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ABSTRACT 

There are an increasing number of desktop decision support systems (DSS) gen- 
erators available which can assist a manager in making decisions. The low cost of these 
packages also make them ideal instructional tools in academic courses covering decision 
analysis. Using literature review, surveys, correspondence and program inspection, this 
thesis demonstrates the features which are required of a good DSS as they relate to three 
potential uses; production, education, and demonstration. 

This thesis discusses the characteristics a prospective user should consider when 
selecting a DSS. These characteristics include features such as the user interface, data and 
modeling support systems and the level of support available from the vendor. Following 
this, the thesis reviews the “state of the art” in currently available programs. 

The programs reviewed in this thesis are easy to use and provide valuable tools for 
decision making. The programs lack in their ability to import and export data to other ap- 
plications which limits their usefulness in a production setting, however, desktop DSS of- 
fer managers a sophisticated, yet easy to use, application which can improve decision 
making and benefit organizations at all levels. 



V 



' ■> 





i: 



TABLE OF CONTENTS 



L INTRODUCTION 



1 



BL BACKGROUND: DECISION MAKING AND DSS 3 

A. DECISION SUPPORT 3 

1. Benefits 3 

2. Limitations 4 

B. DECISION MAKING 5 

1. Decision Environment 5 

2. Decision Strategies 8 

a. Optimizing 8 

b. Satisficing 9 

c. Quasi-Satisficing 9 

d. Selection by Elimination 9 

3. Decision Process 10 

C. RELATED TECHNOLOGIES 10 

1. Executive Information Systems 10 

2. Group DSS. 11 

3. Expert Systems 11 

4. Software 11 

D. SUMMARY 12 

EDL DESCISION SUPPORT SYSTEM FEATURES 13 

A INTRODUCTION 13 

B. METHODOLOGY 13 

C GENERAL DSS CONSIDERATIONS 15 

1. Problem Type 15 

2. What Aspect of DSS Modeling Does the Product Address? 15 

3. Technical Considerations 16 

D. DSS COMPONENTS 17 

1. User Interface Management System ...18 

a. Control Mechanisms 18 

b. Input Representation 21 

c. Output Representation 22 

2. Data Management 26 

a. DBMS Nfanagement Functions and Version Management 27 

b. Data Import and Export 27 

3. Model Management 28 

E. SUPPORT 29 

1. Vendor Support 29 

2. Training 30 

R SUMMARY 30 



IV. DSS USE IN PRODUCTION, EDUCATION, AND DEMONSTRATION ENVIRONMENTS ..33 



A INTRODUCTION 33 

B. PRODUCTION USE 33 

1. Survey Responses 34 

2. Software Selection 34 

3. Application Features 35 

4. Decision Support Features 35 



vii 



5. Support and Training Features. 36 

C. EDUCATIONAL USE 36 

1. Survey Responses 37 

2. Software Selection 37 

3. Application Features 38 

4. Decision Support Features 39 

5. Support and Training Features. 39 

D. DEMONSTRATION 40 

E. VARIATIONS ACROSS USE CATEGORIES 40 

1. Application Features 41 

2. DSS Features 41 

3. Support and Training 41 

F. SUMMARY 42 

V. DESKTOP DESCISION ANALYSIS SOFTWARE 43 

A INTRODUCTION 43 

B. TECHNICAL CONSIDERATIONS 43 

C. USER INTERFACE MANAGEMENT SYSTEM 44 

1. Building Decision Problems 44 

2. Outputs 48 

3. Macros and Programming. 52 

D. DATA MANAGEMENT SYSTEM 52 

E. MODEL MANAGEMENT SYSTEM 54 

F. GROUP FEATURES 55 

G. SUPPORT 55 

H. SPREADSHEET ADD-INS 57 

VL CONCLUSIONS AND RECOMMENDATIONS 61 

A. CONCLUSIONS ; 61 

B. RECOMMENDATIONS 61 

LIST OF REFERENCES 63 

BIBLIOGRAPHY 65 

APPENDIX A - METHODS FOR DECISION MAKING 67 

APPENDIX B - TECHNICAL FEATURES SUMMARY 75 

APPENDIX C - EDUCATOR SURVEY 89 

APPENDIX D - VENDOR SURVEY 93 

APPENDIX E - PRODUCTION USERS SURVEY 97 

APPENDIX F - SURVEY RESPONSES - EDUCATORS 101 

APPENDIX G - SURVEY RESPONSES - VENDORS 109 

APPENDIX H - SURVEY RESPONSES - PRODUCTION 117 



APPENDIX I - COMPARISON OF USER CONSIDERATIONS 125 

INITIAL DISTRIBUTION LIST 127 



IX 



I 










X 



ACKNOWLEDGEMENTS 

I wish to thank Professors Bhargava and Sridhar for their guidance, wisdom, and pa- 
tience while working on this project. They made it a valuable and enjoyable learning experi- 
ence. Mr. David Hoflfer, of TreeAge Software, and Mr. Ed Robins, of Arlington Software, 
provided valuable insight into how designers of DSS software saw the field. Both offered de- 
tailed answers to questions which were often open ended, as well as points of contact in the 
production arena. 

I also wish to thank my wife, Susan, for her patience and support during the many 
hours of work required to prepare this thesis. Lastly, thanks to my parents. Captain Carl 
Herrick and Mrs. Anne Herrick for their encouraging me to pursue education and their 
support during my military career. 



XI 



I. INTRODUCTION 



Decision support systems (DSS) have been available for a number of years. Tradi- 
tional systems were typically large mainframe applications developed for a specific pur- 
pose such as airline reservations or inventory control. In recent years, with the prolifera- 
tion of desktop computers, a growing number of DSS generators have become available 
that enable the manager to easily tailor applications to specific and non-recurring deci- 
sions. Additionally, the low cost of some packages makes them ideal instructional tools in 
academic courses that teach decision analysis. 

The objective of this research is two fold. This thesis will attempt to establish the 
features and characteristics a prospective user should consider when selecting a DSS gen- 
erator. Since the appropriateness of a given DSS depends on its intended use, we will 
consider three potential uses; educational, production, and demonstration. In each of 
these environments different features of a DSS tool are desirable. A DSS used in a pro- 
duction setting would be more useful if it supports linkage with databases of various for- 
mats and sizes, whereas, a DSS used in the classroom would likely use only small demon- 
stration data sets which can be constructed as needed. For demonstration use, power and 
flexibility may be less important than the ability to quickly build a working model. Certain 
features such as Windows and mouse interface, online help, and logical construction of 
decision problems are desirable in all DSS packages. 

Second, having determined what features are desirable in a DSS, this thesis will re- 
view some of the most popular packages currently available detailing their specific advan- 
tages and shortcomings. Due to the rapidly evolving nature of computer software, this 
thesis will avoid specific recommendations for use of software packages, focusing instead 
on what issues the prospective user should consider when selecting a DSS. 

Chapter II presents an overview of decision support systems, decision processes, 
and methods for decision making. Readers who are conversant with DSS can disregard 
this chapter. It is included as background material for those less familiar with DSS. 
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Chapter HI discusses the important features which a prospective user should con- 
sider when selecting a DSS. Characteristics included features relating to the three compo- 
nents including the user interface, the database management system and the model man- 
agement system. Desktop DSS generators are most capable in the user interface area, 
making effective use of the Windows and mouse interface of a PC. Most desktop systems 
lacked extensive database and model management features. 

Chapter IV discusses considerations as they relate to specific uses. Different DSS 
features are more important to some users than others. In a production setting, DSS are 
perhaps more practical for non-recurring strategic decisions. The limited exchange capa- 
bilities of today’s products make them less suited for recurring decisions which may rely 
on external data. DSS can make excellent supplements to classes which teach DSS. They 
can be used either to teach DSS or to demonstrate interactively the concepts of decision 
trees or multi-criteria decision making. 

Chapter V highlights some of the more popular packages based on the considera- 
tions developed in Chapters HI and IV. Chapter VI offers conclusions and recommenda- 
tions for further research. 
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II. BACKGROUND: DECISION MAKING AND DSS 



A. DECISION SUPPORT 

Decision support is more than option selection. In practice, decision support con- 
sists of gathering information, prioritizing, filtering, the identification of decision options, 
the evaluation and implementation of options, and the evaluation of the decision process 
itself, which can occur before, during or after the decision making cycle. [Andriole, 1989] 

Decision support systems are defined as interactive computer programs that utilize 
analytical methods, such as decision analysis, optimization algorithms, program scheduling 
routines, and so on, for developing models to help decision makers formulate alternatives, 
analyze their impacts, and interpret and select appropriate options for implementation. 
[Adelman, 1992] 

In thel970’s, decision support systems were written to support specific recurring 
problems and did not support reuse. These programs were typically custom designed for 
one application. In thel980’s, DSS generators were introduced which allowed a user to 
custom tailor an apphcation to a given decision. These programs typically lacked in vari- 
ous capabilities such as their user interface or data handling capabilities. 

The decreasing cost of hardware and the increasing power of software in 
thel990’s make it possible for decision makers to use sophisticated problem-solving tech- 
niques. Off-the-shelf DSS generators are much more affordable than they were just a few 
years ago. Many generators are now available for personal computers, thereby decreasing 
the implementation costs and problems. They make use of the user fiiendly Windows in- 
terface and offer increased data handling capabilities which make them more appealing to 
the typical manager with a limited management science background. 

1. Benefits 

There are clear commercial successes from the use of DDS. For example, a Cana- 
dian utihty company uses a desktop DSS to determine when to deploy repair crews during 
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minimally stafifed periods. A southwestern power company used a desktop DSS to select 
among various options for increasing power generation capability. 

Improved organizational effectiveness can occur in many ways, such as through 
decreased personnel costs, greater access to expert knowledge, or improved decision 
making [Adelman, 1992]. Perhaps the greatest benefit of using a desktop DSS is not 
having a software package to solve the problem at hand but forcing the decision maker to 
define alternatives and work through a decision lo^cally. Decision analysis is the disci- 
pline of evaluating complex alternatives in terms of values (what we care about) and un- 
certainty (what we know and do not know). Experienced decision analysts and educators 
stress that the benefits of decision analysis are insight into how the defined alternatives 
differ from one another, and generating suggestions for new and improved alternatives. 
Too many critics stress the use of numbers to quantify subjective values and uncertainties 
without realizing the power of quantitative analysis for generating qualitative insight 
[Booed, 1996]. 



2. Limitations 

In spite of the successes, the "state of the art" has not matched the "state of the 
expectation." The fact is that many decision support systems are simply not used. Ven- 
dors have vested interests in overselling, and users are inclined to want to believe that a 
solution to all their problems can be found on one or two floppy disks. Andriole [p. 7] 
points out, that the state of the art of decision support systems technology is unbalanced 
and evolving [Andriole, 1989]. 

Managers do not use models as much as they could. Models require accurate data 
that may be time consuming and costly to produce. Even if a good model is developed 
and accurate data obtained, a manager may not understand it, preferring their own simple 
analysis to a complicated model. Lastly, even when managers do use a model and imple- 
ment the results, they may not realize that this is what they have done. 

In order to utilize such supporting software effectively, the user needs some under- 
standing of the fundamental concepts of models and of decision support concepts. The 



4 



existence of user friendly modeling makes it easy to create bad models or to misuse good 
models [Young, 1989], 

B. DECISION MAKING 

Decision making is a basic skill required in any managerial position. It is only in 
the last 25 years, however, that management theorists have focused on decision making as 
a science. This section gives an overview of decision making, the importance of the envi- 
ronment in which the decision is being made, several strategies for making decisions, and 
the decision making process. 

1. Decision Environment 

There are a number of ways to view the environment in which a decision is made. 
Of primary importance in establishing environmental considerations is at what level in the 
organization the decision is being made. Figure 1 shows the three principle levels of deci- 
sion making, strategic, tactical and operational. 




Figure 1 Levels of decision making 

At the strategic decision making level, top executives develop overall organiza- 
tional goals, strategies, policies, and objectives through long-range strategic planning. 
They also monitor the strategic performance of the organisation and its overall direction. 
At the tactical level, middle level managers develop short and medium range plans and 
budgets, and specify the policies, procedures, and objectives for units of the organization. 
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They also acquire and allocate resources and monitor the performance of organizational 
units. At the operational level, supervisory managers develop short range planning de- 
vices such as production schedules. They direct the use of resources and the performance 
of tasks according to established procedures and within budgets established for them. 

The level at which the decision is being made typically dictates the environmental 
factors which influence decision making. These factors include the type of stimuli^ the de- 
gree of coordination, the nature of the task, and the degree of uncertainty. 

The type of stimuli refers to why the decision is being made. At a higher level, the 
stimulus is more controlled and may involve taking advantage of an unforeseen opportu- 
nity. At the lower level, decisions are more focused on solving urgent problems such as 
allocating personnel and resources. 

The requirement for coordination increases at higher levels in the organization. At 
the lower levels of an organization, decisions tend to be more individual such as a unit su- 
pervisor planning the allocation of resources for a short time frame. At a higher level, ex- 
ecutives may make group decisions using data pulled from many sources both inside and 
outside the organization. 

Uncertainty in decision making also increases as the organizational level increases. 
This is only logical. At the lower levels of an organization, decision makers face some- 
what routine decisions using limited amounts of data. At strategic levels executives are 
faced with decisions which they have never made before and with data pulled from multi- 
ple internal and external sources. 

Nature of task refers to whether the task is unstructured, semi-structured, or 
structured. Structured decisions involve situations where the procedures to follow when a 
decision is needed can be specified in advance. Therefore, such decisions are structured or 
programmed by the decision rules developed for them. A structured decision may involve 
what is knovm as a deterministic or algorithmic decision. In this case, a decision’s out- 
come can be determined with certainty if a specified sequence of activities (an algorithm) 
is performed. An example of such a system is the Navy’s Uniform Inventory Control 
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Point (UICP) system which runs at the Navy Inventory Control Point. This system re- 
ceives a requisition for material from a Navy unit, and then passes the requisition to a sup- 
ply point for material issue based on such factors as the on-hand balance of the item at the 
various supply points. The rules regarding filling outstanding requisitions are programmed 
into UICP’s BOl module and will result in a predictable decision being made for a set 
group of inputs. 

Unstructured decisions involve decision situations where it is not possible or desir- 
able to specify in advance most of the decision procedures that will follow. Many decision 
situations in the real world are unstructured because they are subject to too many random 
or changeable events or involve too many unknown factors. At most, many decision 
situations are semi-structured. Decisions involved in starting a new repair line at a Naval 
Aviation Depot, for example, would range from unstructured to semi-structured. The 
many unknown factors involved would require a less structured approach leading to sub- 
jective judgements by managers. Information systems can support such decisions by pro- 
viding the ability to make ad hoc inquiries for information in databases and the ability to 
reach a decision in an interactive process using a DSS. 

Decisions are also broken down between nonrecurring decisions, also called ad 
hoc, and recurring, also called institutional. The type of decision being made dictates 
many of the features which are required of the DSS as shown in Table 1. 
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Ad hoc vs. Institutional use: 




Institutional 


Ad Hoc 


Decision occurrences for a decision type 


Many 


Few 


Decision types 


Few 


Many 


Number of people making decisions of same type 


Many 


Few 


Range of decisions supported 


Narrow 


Wide 


Range of users supported 


Narrow 


Wide 


Specific data known in advance 


Usually 


Rarely 


Problems recurring 


Usually 


Rarely 


Importance of operational efficiency 


High 


Low 


Duration of specific type of problem being addressed 


Long 


Short 


Need for rapid development 


Low 


High 



Table 1 [From Sprauge, 1996] 



2. Decision Strategies 

There are several avenues of approach that a decision maker can utilize in order to 
reach a decision. Different people will use different strategies at different times for differ- 
ent kinds of decisions. Which strategy to select depends on factors such as what is the 
information processing requirements for each decision making strategy and which strategy 
the decision maker prefers. Several strategies will be discussed including optimizing, sat- 
isficing, and selection by elimination. 

a. Optimizing 

The goal when optimizing is to select the course of action with the highest 
payoff Using this technique requires the decision maker to estimate all costs and benefits 
of every viable course of action. In even a simple decision making scenario, optimizing 
can be an overwhelming task. In many cases, there is too much information to process 
and too many variables to consider simultaneously. This can result in a high cost in time, 
effort and money. Additionally, optimization of stated objectives may result in sub- 
optimization on unstated, less tangible objectives. As a result of the difficulty with the 
optimization technique, decision makers fi'equently do not consider all possible alterna- 
tives, do not consider all objectives and criteria or place more weight on intangible objec- 
tives. 
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Many desktop DSS generators perform "comparative optimization". The 
user declares the alternatives, and the programs helps the user compare them. Standard 
optimization tools (for linear programming) generate the alternatives from an abstract de- 
scription of the constraints and test them. 

b. Satisficing 

Due to the complexities inherent in optimizing, decision makers often sat- 
isfice. They choose courses of action that are “good enough” that meet a certain minimal 
set of requirements. Multiple criteria are often used, but the “satisfactoriness” of an out- 
come is the decisive factor. Often, uncertainty about the best choice makes decision mak- 
ers gravitate towards a more conventional second best choice. 

c. Quasi-Satisficing 

Quasi-satisficing is usually used in cases of crisis management or when the 
decision involves someone else’s problem. It is applicable when there are fewer objectives 
that need to be met or when very few alternatives are generated. Quasi-satisficing can be 
summarized as “do something useful without deliberating about all other alternatives," or 
“do what we did the last time if it worked, or the opposite of if it did not." 

d. Selection by Elimination 

Selection by elimination involves eliminating alternatives that do not meet ^ 
the most important criterion. Decision making using this strategy becomes a sequential 
narrowing down process. The disadvantage of this technique is that if improper weights 
are assigned to criteria, a “better” alternative might be eliminated early on. In addition, for 
more complex problems, this process might still leave the decision maker with a large 
number of alternatives. 



3. 



Decision Process 



The decision making process is generally considered to consist of three phases: 
intelligence, design, and choice. Intelligence involves the identification of a problem that 
requires a decision. Design involves the creation and evaluation of alternative courses of 
action. Choice is the selection of a course of action. 

DSS typically addresses a specific problem that has been identified. Thus, it is of- 
ten unnecessary to support the intelligence phase, especially for decisions at a lower lev- 
els. 

The ability to support the design phase of decision making is the true test of DSS. 
The core of any DSS is the model base which has been built to analyze a problem or deci- 
sion. The primary value to a decision maker of a DSS is the ability of the decision maker 
and the DSS to explore the models interactively as a means to identify and evaluate alter- 
native courses of actions. This is of tremendous value to the decision maker and repre- 
sents the DSS capability to support the design phase. [Sprague, 1996] 

The choice phase of decision making is the most variable in terms of support fi"om 
DSS. Although a custom designed DSS can be programmed to make decisions, DSS gen- 
erators do not make decisions y them selves. They do however, present the user with a 
detailed analysis of the factors influencing the decision. In this regard they support the 
choice phase. 

C. RELATED TECHNOLOGIES 

There are a number of other types of programs that are similar to decision support 
systems and which are commonly grouped with DSS when teaching decision support or 
thinking about decision making. These programs include executive information systems, 
expert systems, and group DSS. 

1. Executive Information Systems 

Executive Information systems (EIS) typically are used to obtain and provide top 
managers with status information from various internal and external sources. Information 
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is presented in a wide variety of forms (textual, graphical, tabular) often on the same 
screen. An example of such a system was developed by Sherwin Williams Company. 

Their system, developed over the course of two years, was designed to replace a paper re- 
porting system which ran from individual stores, through the regional management to the 
corporate headquarters. The new system uses a graphical interface and allows managers 
at all levels to review sales data of units under their control. [Booker, 1993] 

2. Group DSS 

One of the most popular forms of decision support systems being developed today 
is the Group DSS (GDSS). Group DSS focus on group activities such as discussions and 
meeting. The systems are designed to provide computer support for groups of people 
who work on collaborative projects. GDSS can play an important role in today’s chang- 
ing management environment. They can keep people in better touch with each other, sup- 
port broader spans of control, and directly connect members of different organizations. 
GDSS includes a wide variety of products and are also called groupware, collaborative 
work support systems. [Sprague, 1996] 

3. Expert Systems 

Expert systems employ heuristics and quahtative reasoning to simulate the behav- 
ior of an expert, based on knowledge accumulated by experts. An example of an Expert 
System would be a program to help a medical professional diagnose an injured patient or 
to help a technician trouble shoot a piece of equipment. 

4. Software 

There are a number of software packages available which executive information 
systems, group DSS, expert systems, as well as forecasting. EIS are available from a 
number of companies. Comshare from Commander EIS or Pilot from Command Center 
allow users to develop custom EIS designed to take advantage of unique concerns of 
management. Group DSS are available from numerous vendors including Team Expert 
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Choice by Expert Choice and GroupSystems developed by the University of Arizona. 

Like EIS, Expert systems are frequently custom designed for a specific organization. De- 
velopment tools include programs such as VP Expert, SuperExpert, and KnowledgePro as 
well as programming languages such as C-H-. Forecasting programs are available includ- 
ing such titles as. Minitab Statistical Software by Minitab Inc., SAS Software from SAS 
Institute Inc., Statgraphics Plus for Windows by Statistical Graphics Corp., and SPSS 7.5 
for Windows by SPSS Inc. 

Reviews of these types of program can be found in various information technology 
related journals including OR/MS, Datamation, and others. 

D. SUMMARY 

This chapter has given a brief overview of decision support systems, their benefits 
and their limitations. Traditional DSS differs from DSS generators in that the later can be 
easily tailored to specific situations. Appendbc A provides additional information on vari- 
ous methods of decision analysis including decision making under uncertainty and decision 
making with multiple criteria. Readers unfamiliar with these techniques should review this 
appendbc prior to proceeding to the next chapter. 



III. DESCISION SUPPORT SYSTEM FEATURES 



A. INTRODUCTION 

When evaluating decision support systems, it is important to remember the obvi- 
ous, which is that the overall aim of these systems is to improve the effectiveness of the 
organization using them. The first requirement to fulfilling this aim is to determine the 
correct type of package for the users need. After a review of the methodology used for 
this thesis, this chapter discusses some of the important characteristics on DSS generators 
may differ. Issues include such things as the technical considerations and the components 
of a DSS. Chapter IV discusses issues tailored to different categories of users. 

B. METHODOLOGY 

To determine which features are important in DSS selection the research for this 
thesis involved a review of current literature on DSS. Material was drawn from texts, 
magazines, journals and the World Wide Web. Most texts and papers written about DSS 
have similar recommendations regarding writing or selecting DSS packages. 

In order to classify features important to the three categories of DSS users, an in- 
formal brainstorming session was undertaken to group, classify, and further develop the 
items on the list developed from literature review. From this fist, we developed a survey 
which was sent to over 100 educators and professionals, as well as to DSS software com- 
panies. A separate survey was developed for production users, educational users, and 
vendors. Names were drawn from the online database of the Institute for Operations Re- 
search and Management Science (INFORMS) and from a search of the World Wide Web. 
Responses were received from 35 percent of those surveyed. The aim of the survey was 
to validate and refine the list obtained through brainstorming and to gain insight into other 
features DSS users felt were important. The surveys asked users to assign weights to the 
features of DSS, assigning a higher score to features they felt were most important. Addi- 
tional dialog was held with DSS vendors and some users to explore questions raised by 
the survey. Detailed, anonymous responses to the survey are shown in Appendices F, G, 
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and H. Appendix I provides comparison between the average scores on the detailed sur- 
veys. 

The survey consisted of seven questions. The first two questions were designed to 
gather background data. Educators were asked about the level at which they taught DSS 
and the role of DSS software in their class. Designers were asked about the number of 
copies of DSS software the company had produced. All users, production and educa- 
tional, were asked about packages used, or rejected for use, and the reasons for doing so. 
Designers were also asked who they saw as their primary customers and how they best 
saw DSS software being used for strategic or operational use. 

Question three regarded criteria to be considered when selecting DSS software. 
Survey recipients were asked to allocate 100 points between five criteria including the im- 
portance of the length of time required to learn to use the software, the ease of use, the 
availability of versions of the software to operate on multiple platforms, the ability to use 
the software in a network environment, and the cost of acquiring the software. 

Question four asked questions related to application features. Again, survey re- 
cipients were asked to allocate 100 points among ten criteria including divided among 
features of the user interface, the data handling system, and the model system. 

Question five asked recipients to rate the importance of criteria related to training 
users to use the program. Recipients were asked to allocate 100 points between the im- 
portance of sample exercises, online tutorials, explanations of results, and tips and in- 
sights. 

Question six asked recipients to rank the importance of specific DSS considera- 
tions included in a software package, including the ability to modeling uncertainty, to han- 
dle subjective data, to work with utility functions, and to measure risk and sensitivity. 

Lastly, question seven asked the users' questions related to support issues includ- 
ing the availabiUty of FAQ files, discussion groups, mail hsts, technical support via phone 
or email, local support, and documentation. 
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c. 



GENERAL DSS CONSmERATIONS 



This section discusses the general considerations for selecting a DSS. Criteria in- 
clude what type of decision the user is trying to solve, what aspects of modeling a par- 
ticular product addresses, and the technical requirements of the system, 

1. Problem Type 

What DSS problems does the product address? There is no one DSS package that 
is well suited to all decision problems. Broadly, the DSS packages reviewed for this thesis 
can be grouped into two areas, decision making under uncertainty and decision making for 
multi-criteria problems. Decisions involving uncertainty are typically solved using influ- 
ence diagrams and decision trees. Decisions with multiple criteria are solved using the 
analytical hierarchy process. It is the responsibility of the potential user of these packages 
to determine which approach is most appropriate. A third category of programs that we 
reviewed, the spreadsheet add-in, does not fit into these two traditional areas. Though 
not technically a DSS, these packages are very useful tools for the decision maker. 

2. What Aspect of DSS Modeling Does the Product Address? 

In spite of the differences between applications, it is possible to identify certain 
common aspects of DSS modeling projects. Most attempts to use DSS software involve, 
for example, defining model components such as decision variables, objectives and con- 
straints; gathering data required by the model; solving model instances; reporting and 
analyzing results; managing multiple model versions and data scenarios. 

Independent of the specific software package being used, in order to create a deci- 
sion analysis model, it is necessary to create the model structure and obtain the data to 
populate the model for computation. In some instances, trained analysts provide the 
model structuring support, however, the ease of use of these packages allows anyone to 
easily build relationships models. Some DSS start with an easy to use brainstorming fea- 
ture and then progress to a decision support model. 
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Once the structure and numbers are in place, the analysis can begin. Much more is 
involved than computing the expected, weighted utility of each alternative. If the process 
stopped there, decision makers would not gain much insight into the problem being 
solved. Decision makers must examine the sensitivity of the key criteria and weights and 
risk preference parameters. As part of the sensitivity analysis, it is possible to calculate the 
value of perfect information for uncertainties that have been explicitly modeled. [Booed, 
1996]. The more sophisticated DSS should allow the user to perform various forms of 
sensitivity analysis and to investigate decisions in depth. 

The packages reviewed for this thesis are the stand-alone modeling systems, which 
are intended to provide the entire interface to the formulation, solution, and analysis stages 
of modeUng. Some of the models allow import and export of data but they lend them- 
selves only vdth difiBculty to automated processes such as one that may be written with 
Delphi or Visual Basic. 

3. Technical Considerations 

The technical considerations involved in selecting a DSS are fairly straight forward 
and will only be discussed briefly. 

•Operating systems, memory requirements, and printer support -- In most in- 
stances, the choice of operating systems and the memory required for a desktop DSS is . 
dictated by a system that is already in place. Chapter IV discusses specific requirements of 
desktop systems. Appendix B lists requirements for currently available packages. 

•Security -- Users should consider the sensitivity of the data that they are working 
with, especially if the program is being used in a network environment. Additionally, in- 
tegrity may become an issue if more than one person is working on the model or the data- 
base. 

•Group versus individual — Many DSS which are being introduced today are de- 
signed to support group interaction. These packages contain voting tools and features 
which allow multiple people to combine their input on decision making. This thesis con- 
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centrates on packages designed for individual use. Where group features are present, they 
are noted. 

•Process automation support -- More sophisticated users may require the ability to 
link the decision making abilities of a DSS to the analytical capabilities of a spreadsheet or 
to a database. A few users surveyed discussed the desire to be able to use Object Linking 
and Embedding (OLE) to control a decision process. 

D. DSS COMPONENTS 

DSS technology consists of three sets of capabilities, the User Interface Manage- 
ment System (UIMS), the Data Base Management System (DBMS) and the Model Man- 
agement System (MMS). Figure 2 is a depiction of the relationship. The UIMS, also 
called the Dialog Generation Management System (DGMS) manages the interface be- 
tween the user and the rest of the system. The DBMS and the MMS contain the neces- 
sary functions to manage the data base and model base, respectively. The characteristics 
of each section are discussed in greater detail later in this chapter. 
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Figure 2 The components of DSS. 

A good DSS should have balance among the three capabilities. It should be easy 
to use to allow non-technical decision makers to interact with the system, it should have 
access to a wide variety of data, and it should provide analysis and modeling in a variety of 
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ways. [Sprague, 1996] Unlike the early DSS generators which lacked in one or more of 
these areas, the current desktop systems oflFer a reasonable balance between these three 
components. 

1. User Interface Management System 

To a large extent, to the user, the User Interface Management System is the DSS. 
The sense of the general character of the software, but not the detailed operational struc- 
ture, is obtained through interaction with UEMS. The power and scope of DSS function- 
ality can only benefit a user if it is accessible and comprehensible. Ease of use and under- 
standing are determined largely by the character of the interface. Similarly, the more a 
DSS can do, the more difficult it is to provide an interface that is easy to use and under- 
stand. [Young, 1989] The UIMS is characterized by the control mechanisms of the pro- 
gram, the quality of the display, and the dialog which is held between the system and the 
user. 



a. Control Mechanisms 

(1) Mouse and Windows Support -- Respondents to the thesis sur- 
vey placed strong emphasis on the necessity of having mouse and Windows support for 
DSS packages. The graphical abilities of Windows make available a number of useful 
features such as dynamic sensitivity analysis which will be discussed later. Though most 
programs written today utilize the Windows interface, some provide more flexibility than 
others. For example, only a few programs make use of the right mouse button which 
opens a pop up menu showing the most appropriate features depending on where the 
mouse is pointing. This can be a time saving feature for the experienced user. 

(2) Visualization and Ingredients of a Good Display -- Proper dis- 
play design is critical. A number of studies have shown that tasks involving visual search, 
counting, noting of display changes and information extraction are adversely affected by a 
high numbers of items displayed on a screen. The more information displayed on a screen. 
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the greater the chances for error, misperception, frustration and eventual abandonment of 
the system. Ingredients to look for when evaluating a good display include: 

• Orderly, clean, clutter free displays. 

• Obvious indication of what is being shown. 

• Expected information where it should be. 

• Clear indication of what relates to what. 

• Plain simple English. 

• Clear indication when an action could make a permanent change in the data or 
model design [Andriole, 1989]. 

Many users find color input and output routines initially pleasing, 
however, research suggests that unless color is used carefully, its initial appeal will shortly 
be replaced with fioistration. Many color displays violate some basic rules of presentation. 
First, too many colors often appear. Some use color inconsistently. Some use offensive 
combinations. One often wonders if the authors of software use color to improve per- 
formance or use it simply because color display devices have dropped in price. Figure 4 
shows a display which violates most of the rules for appropriate guidelines for the use of 
color. Some considerations users should look for when evaluating the quality of a color 
display include: 

• Use color as a formatting aid to relate or tie fields into groupings, emphasize 
important fields and relate fields that are spatially separated. 

• Use color as a visual code to identify kinds of data, sources of data, status of 
data. 

• For emphasis and separations use contrasting colors such as red and green. 

• To convey similarity use similar colors such as orange and yellow. 

• Use colors consistently within and across systems. 
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Figure 3 This output uses color for the sake of color. There is no meaning 
behind the colors selected in the output. [From Decide Right by Avantos] 

The improper use of color will undermine the human-computer 
communication. [Andriole, 1989] 

(3) Dialog management -- Over the years a number of different 
dialogues have been used for DSS. Dialog types include menues, query language, func- 
tion keys and form filling. These dialog types vary in the complexity they involve as well 
as the flexibility they provide. 

Menus are fi'equently used for command construction as well as 
data-base search. Pull down or pop up menus are frequently used to drive Windows based 
programs such as the ones review for this thesis. Query Language is frequently found in 
older programs. In this type of program, the user enters questions or data base access re- 
quests and the system produces a report. This is acceptable for a well-trained user, oth- 
erwise it can be error prone and frustrating. Query language is also used to access the 
help features of many Windows based programs allowing the user to enter the topic on 
which help is desired. Function Keys provide a short cut to frequently used commands and 
are found in some of the desktop DSS tools. Function keys provide flexibility for more 
experienced users of a program. Form filling presents a form with fill-in blanks which the 
user files in. This type of dialog is fi'equently used to enter data or weightings. 
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Dialog involves complexity versus flexibility trade-offs. For exam- 
ple, while the question-answer approach to data entry is simple and often appropriate for a 
novice user performing well structured tasks, it does not provide flexibility beyond what 
was plaimed by the systems designers. Menu oriented systems impose the same kind of 
structure. In contrast, command languages place the user more in control but require ad- 
ditional knowledge to use the systems. [Sprague, 1996] 

Flexibility is a measure of the number of ways in which a user can 
accomphsh a given function and can be achieved by providing a large number of com- 
mands. There is evidence that less experienced users tend to utilize known methods for 
solving a problem even when the system provides less cumbersome methods. There is 
also evidence that more flexible dialogues degrade performance of relatively inexperienced 
users. 

Complexity is a measure of the number of options available to the 
user at a given point in the dialogue. Low complexity can be achieved by using few com- 
mands or by partitioning the commands so the user selects from a small set at any given 
time. [Andriole, 1989] 

Most Windows based programs, including DSS generators, offer 
some degree of flexibility. A particular task may be performed in a number of ways such 
as with a pull down menu, a pop up menu brought up by double clicking an item, with a 
control key sequence or function key. This allows the user to perform a task in the man- 
ner in which they are most comfortable. 

b. Input Representation 

How are DSS problems expressed? The simplest way to describe a DSS 
problem to a computer system is to input a separate and explicit description for each indi- 
vidual variable or constraint (or both). Depending on the decision problem, various types 
of input data may be available including absolute or relative data. 

In some instances, the user may have absolute data derived from experi- 
ment and testing. In selecting a new fighter aircraft, the prospective buyer would have ex- 
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plicit knowledge in advance regarding the speed of the competing aircraft and the utility 
derived from various speeds. 

Programs which accept absolute data take the data in a variety of forms in- 
cluding; explicit data (e.g. mach 1.4) or as statistical values. The decision tree programs 
and a few of the hierarchy base programs accept data in this manner. Where explicit data 
values are entered in hierarchical programs, the user also has the opportunity to enter util- 
ity values for the various data points. Many of the programs will allow the user to enter 
statistical data in place of actual numbers. This is useful when using experimental data and 
results in output which represents a range of likely outcomes. 

In contrast, in selecting a new car, the buyer would not have an explicit 
measurement of his preference for one car’s style over another. He would probably know 
that he thinks that one vehicle is attractive while the other isn’t or he may know that he 
prefers one design over another. In this case he has relative data; he knows the relative 
preference of one item over another, even though no specific scores can be assigned. Pro- 
grams which use relative data, require the user to choose a preference of one item over 
another. This may be done by using a slide bar, by manipulating icons, or by entering nu- 
merical preferences. This type of data entry may be most useful when the user wants to 
make a quick decision without taking the time to gather detailed data. 

c. Output Representation 

How are results viewed and analyzed? DSS packages produce a variety of 
outputs which are useful in decision analysis. All programs present their recommendations 
in a graphical format while some also produce tabular data. Graphic displays are fre- 
quently much better communicators of information than alphanumeric. 

All of the multi-criteria based programs produced a variety of bar charts 
showing the overall utility of the different output variables. Figure 4 shows a stacked bar 
chart which provides details concerning which variables contributed most to the recom- 
mendation. 
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Figure 4 Multi-criteria programs generate a bar chart allowing the user 
to evaluate the contribution from various input variables. [From LDW 
by Logical Decisions] 



A few programs will produce a greater variety of outputs. Figure 5 shows 




scatter diagram which allows a quick comparison between two input variables. 



Figure 5 A scatter diagram compares two variables graphically. 

[From LDW by Logical Decisions] 

Programs which perform decision making under uncertainty typically pro- 
duce a probability distribution chart as shown in Figure 6. This allows the decision maker 
to determine the likelihood of the various outcomes. 
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Figure 6 Programs using decision making under uncertainty produce prob- 
ability distribution charts. The chart on the left shows the cumulative dis- 
tribution. [From DATA by Tree Age] 

(1) How are problems solved? — The users of DSS need a flexible 
mechanism that displays results quickly and interactively. Whether for demonstration or 
production use, a DSS must do more than simply display result values returned by the 
solver. Programs should offer some degree of insight into how the problem was solved. 

Perhaps the most useful feature offered is the ability to perform 
sensitivity analysis. Sensitivity analysis shows the change in output due to a given change 
in one or more input variables. In the thesis survey of what users of DSS, designers, and 
educators felt were important features for DSS packages to have, respondents gave an av- 
erage of 30 out of 100 points to sensitivity analysis, the most heavily DSS related 
weighted factor. A typical sensitivity analysis graph is shown in Figure 7. 
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Figure 7 Sensitivity graph shows the change in an output variable when an 
given input variable is changed, [From LDW by Logical Decisions] 

Nearly all decisions involve situations which are probabilistic in that 
the behavior of the system being modeled cannot be predicted with certainty because a de- 
gree of randomness is present. A probabilistic model attempts to capture the probabilistic 
nature of the system by requiring probabilistic data inputs and by generating probabilistic 
outputs. [Sprague, 1996] DSS users assigned 30 out of 100 available points to the im- 
portance of the ability to model this uncertainty. Some of the programs reviewed allow 
the user to enter a variety of statistical distributions in place of actual data values. LDW 
for instance allows the user to enter a point estimate, normal distribution, uniform distri- 
bution, discrete distribution, piecewise linear cumulative or three point estimate. The pro- 
gram will then show this uncertainty on the output as shown in Figure 8. 
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Figure 8 This chart shows uncertainty as a range of possible outputs in a 
decision where a normal distribution was assigned to one of the decision 
criteria. [From Decision Plus by Info Harvest] 

(2) Explanations Of Recommendations -- Survey respondents gave 
considerable weight to the importance of a program providing insight into its recommen- 
dations. According to the designer of one of the popular packages: 

There are a few key elements to trigger the understanding of a user to the 
process of decision making. This hurdle can be overcome, often with a few 
minutes of proper instruction or guidance within the software. The trick is 
how to achieve this. More sophisticated users must be more aware of how 
to build models to represent a decision or process. Training courses are 
under development to assist both the basic user and the more advanced 
user on how to fully utilize the produce. 

(3) Using the output — In addition to viewing output on the screen, 
some of the programs allow the user to export graphical output to other applications for 
use in other programs such as a word processor. This is important since many decision 
will have to be written up and presented to others. 

2. Data Management 

It is increasingly critical for the DSS to use all the important data sources internal 
and external to the organization. This includes concepts, ideas and opinions that are im- 
portant to decision making. In order to effectively manage data, a good DSS should con- 
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tain features for managing and manipulating data and for keeping track of different ver- 
sions of the data. It is also desirable to be able to import and export data from other ap- 
plications such as databases and spreadsheets. 

a. DBMS Management Functions and Version Management 

Desirable DBMS functions include such things as the ability to manipulate 
rows of data or to extract subsets of a database. The thesis survey asked about the im- 
portance that the program have DBMS functions. Though the survey respondents allotted 
only minimal importance to this feature, it is valuable to have at least minimal DBMS fea- 
tures available if working with large data sets. 

Version management allows users to work with various versions of a data- 
base. This is important when the data is being used by different users or being modified 
for different trials. Again, this feature was judged relatively unimportant by all respon- 
dents but would be valuable. 

b. Data Import and Export 

The survey asked users how important it was for the DSS to be able to im- 
port and export data directly from other apphcations and databases. Surprisingly, corpo- 
rate users and program designers allotted less than 15 points (out of 100 available design 
feature points) to this ability. Educational users allotted only 6 points. This would lead to 
the conclusion that in most cases, desktop DSS users are using the programs to model 
stand alone decisions vice repetitive ones which draw on an external data source. In the 
words of one DSS vendor: 
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Some people seem to use data import/export extensively, but . . . they are in 
the minority. . . . most people do not have the kind of data used by decision 
analysis programs (e g., probability data) already prepared and stored in 
different formats. In most cases, people seem to estimate, calculate, or de- 
rive this information for the specific purpose of building a decision-analytic 
model. And to the extent that such data is being developed solely for this 
purpose, there is no reason to add an extra "layer" of applications in order 
to import the data into a decision analysis program. 

Some specialized groups of users, such as epidemiologists (who can import 
probability data from databases) and financial analysts (who can import fi- 
nancial data fi'om spreadsheets) probably use import/export features with 
substantially greater fi-equency than the overall population of decision ana- 
lysts. This perception is consistent with the theory that such use is tied to 
the availability of preexisting raw data. 

In spite of the lack of emphasis, this is an important feature. Being able to 
import and export data allows users to access corporate databases without having to re- 
enter data into a DSS. As more programs offer data exchange and data becomes more 
widely available, it is logical that DSS users tvill want to be able to extract data from cor- 
porate databases for use in DSS programs. 

3. Model Management 

Models provide the analysis capabilities for a DSS. Using a mathematical repre- 
sentation of the problem, algorithmic processes are employed to generate information to 
support decision making. Among the more heavily weighted features of DSS according to 
respondents to the survey was the ability of the DSS to build large complex models. 

There are essentially two schools of thought regarding the model function of deci- 
sion support systems. One is that the main purpose is to support the decision-maker in 
whatever style of decision-making chosen. The other is that the support, while suiting the 
decision-maker's individual style, must also be based on an appropriate theoretical para- 
digm. [Edwards, 1994] If a user is called upon to defend a decision, it may be more useful 
if the model used is an accepted one such as AHP. In the words of a DSS designer: 
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It is important to be able to recreate the "audit trail" that leads to any par- 
ticular utihty, so the math functions are important. However, this should 

be very easy unless the software uses a proprietary method. 

Several of the multi-criteria based programs use accepted models such as AHP or 
SMART. Others use proprietary models. 

Model management features of a DSS include features as being able to import and 
export model structure from other programs and to maintain control over versions of 
models. 

Versions are most likely to proliferate when a model is being developed. Even 
relatively simple decisions can easily grow into models with hundreds of nodes. Version 
management features allow the user to modify a model and then return to a previous ver- 
sion if desired. This is an important feature, however, it received relatively little emphasis 
from users of DSS software. 

E. SUPPORT 

This section discusses the methods by which a user can receive help when learning 
to use a program and interpret its output. Help can come in a number of forms including 
paper and online documentation, sample exercises and tutorials, or Web based exercises. 
Companies offer various degrees of support from phone and email to full consulting. 

User preferences are discussed in the next chapter. 

1. Vendor Support 

What Aspects of DSS Modeling Does the Vendor Address? While evaluating the 
software available, it is important to determine what the vendors are offering to sell, and 
to whom. In many cases with DSS software there are services available. The software 
may come in any of several forms, and may be addressed to any of several kinds of users. 
Some vendors may not offer a broad enough range of services to meet the users needs, 
while others may have a motivation to sell things that you do not want. All six of the ven- 
dors who responded to the survey offer at least some consulting. At least one of the ma- 
jor DSS producers offers primarily consulting with software being only a sideline. 
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2. Training 

Documentation can come in various forms including paper manuals, help files, em- 
bedded training, and now, tutorials located on the World Wide Web. 

Embedded training is computer based. Far more than help routines, embedded 
training modules permit users to digress for a moment during a problem solving session to 
learn or relearn something with which they may be unfamiliar. They permit users to learn 
as they go and even to decide when they want or need a small refi’esher course. Self 
paced training permits users to learn at their own pace while eliminating scheduling, logis- 
tical and location problems. [Andriole, 1989] 

F. SUMMARY 

This chapter provided an overview of the basic features the potential user of a DSS 
must consider. Features included the type of DSS problem a particular package addresses 
and the technical requirements of a program. DSS packages consist of three components 
including the user interface, the database management system and the model management 
system. The important features of these components are summarized in Table 2 below. 
Desktop DSS are strongest in the user interface area, making extensive use of the Win- 
dows and mouse interface of a PC. Most desktop systems lacked DBMS and MMS fea- 
tures at any more than a rudimentary level. 

The next chapter will discuss DSS as they relate to use in education, production 
and demonstration. 
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DSS Features 

What Problems Does the Product Address: 

• Decision m aking under uncertainty 

• Multi-criteria decision making 

User Interface Management System 

• Mouse and Windows support highly desirable 

• Clean orderly display which makes appropriate use of color 

• Dialog features menus, query or natural language, prompting, or form filling 

• Relative versus absolute data values 

• Bar chart and stacked bar chart outputs 

• Sensitivity analysis and ability to model uncertainty important and add insight 

• Ability of program to explain recommendations desirable 

Database Management System 

• Data import and export desirable especially with large or reciuring data needs 

• Version management desirable 

Model Management System 

• Model import and export desirable 

• Version management important 

Table 2 
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IV. DSS USE IN PRODUCTION, EDUCATION, AND DEMON- 
STRATION ENVIRONMENTS 



A. INTRODUCTION 

The previous chapter discussed some of the issues which affect all users of deci- 
sion support systems. This chapter will look at specific users which have been classified 
into three categories, those who use the programs for production, those who use the pro- 
grams for demonstration, and those who use the programs in education. 

Not surprisingly, there were differences in what educators found important in DSS 
software and what production users found important. We received survey responses from 
six of the leading DSS software designers. One of the questions asked of them was to al- 
locate 100 points between four categories of users: educational, strategic production, tac- 
tical production and demonstration. In this grouping, designers allocated an average of 53 
points to strategic use and 24 points to tactical use while allocating only 1 5 points to the 
importance of educational use. It is not surprising that the responses received from pro- 
duction users of DSS software were more closely aligned with the views of the designers 
than were responses received from educators. Says the designer of one program; 

Our interests are not the same as the academic community - we have no 
theory to defend, only a need to ensure that client requirements are met as 
best as possible and at the highest technical standard. This means under- 
standing how we can help the user in reaching the decision through the 
most appropriate mechanism. 

B. PRODUCTION USE 

A growing number of organizations have developed DSS for applications ranging 
from production control to strategic planning. As discussed in Chapter I, Section B. 1, 
decisions can vary in the degree of structure of the decision making task which they sup- 
port. DSS are being used for one time ad hoc decisions as well as recurring institutional 
uses. 
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Desktop DSS are an ideal tool for managers faced with ad hoc decisions, the type 
managers face frequently. The packages reviewed for this thesis allow the user to quickly 
set up decision scenarios and solve problems. The progranis are also suited for institu- 
tional uses, though they are somewhat limited by their scant data import and export capa- 
bilities. 



1. Survey Responses 

Among the respondents to the thesis survey, five were designers of DSS software 
and seven were experienced users. Although a statistically insignificant number, the inputs 
provide insight on what users of this type of software feel is important and serve to vali- 
date the assumptions made for this thesis. For the purposes of this survey, designers of 
DSS software were grouped with users. Since the designers interact with a number of us- 
ers, their answers represent, to some extent, a broad view of what the typical user expects 
to see in a DSS. The survey asked users of this type of software as well as designers to 
suggest whether they felt the programs were more useful for strategic use or tactical use. 
In every instance, the designers of the software felt the programs were more appropriate 
for strategic use while users were more evenly mixed. In the case of the users, however, 
the small sample size likely skewed the data. The nature of these programs lend them 
more towards strategic use. Appendix H shows detailed responses for production users 
and designers. 

2. Software Selection 

Survey recipients were asked to rank between sbc criteria to consider when select- 
ing DSS software including the length of time required to learn the program, the ease of 
use, the availability of multiple platform support including use on a network, cost and the 
power of the package. Users, as well as designers, ranked the ease of learning and using 
a package as the most important criteria with the power (variety of features, outputs, size 
of models) a close second. The cost of the package received relatively little emphasis. 

The emphasis of designers can be summarized in the words of one: 



We have endeavored throughout our history to offer extremely intuitive 
applications that have enough power to satisfy the most sophisticated us- 
ers. This entails attention to the ease of the user interface, thorough and 
graphical output and the ability to interface with other programs. 

3. Application Features 

Survey respondents were asked to rate between ten applications related features 
broken down into user interface, data handling and modeling features. The details of these 
features were discussed in Chapter n. 

Users assigned an average of approximately 30 points to the importance of a well 
designed user interface including mouse and windows support, a well laid out input inter- 
face and good output representations. All of the packages reviewed support Windows 3.1 
though few specifically support Windows 95. 

Users as well as designers of DSS gave surprisingly few points to the data 
handling features of a DSS. They allotted the most points in this area to 
the abihty to import and export data with little consideration given to ver- 
sion management or the availability of DBMS functions. 

Users assigned a total of 44 points to the model management area with the most 
points being allocated to the importance of being able to build large complex models. 
Model version management earned nine of 100 points fi'om DSS users. None of the pack- 
ages reviewed incorporated version support other than saving files with different names. 

4. Decision Support Features 

The thesis survey asked production users to allocate points between five DSS re- 
lated features including the ability to model uncertainty, use subjective data, enter utility 
functions, determine risk and perform sensitivity analysis. The ability to perform sensitiv- 
ity analysis was considered the most important feature in this area followed by the ability 
to model uncertainty. This only makes sense. In a production environment, it is important 
that a manager not only be able to make a decision but also to understand and explain the 
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decision. Sensitivity and uncertainty analysis provide a valuable tool to perform this fea- 



ture. 



5. Support and Training Features 

The survey asked users and designers to rate the importance of training functions 
including the availability of sample exercises and tutorials and whether the program pro- 
vides explanations and insight. Most respondents felt that the availability of sample exer- 
cises was very important giving that feature 26 points. The availability of tutorials was 
just less important with 18 points. 29 points were allocated to the importance of the pro- 
gram explaining results with 28 points being assigned to the importance of the program 
providing insight into recommendations. 

Production users were asked to rate the importance of six support features includ- 
ing; an online list of frequently asked questions, availability of discussion groups and mail 
lists, support via phone or email, the level of local support and the availability of online 
documentation. The availability of phone or email support was rated as the most impor- 
tant feature. Production users were unconcerned about being able to receive support via 
discussion groups or mail lists. 

C. EDUCATIONAL USE 

The use of technology is changing the expectations of students and faculty. Stu- 
dents who grow up in a technological age vrtll not accept lectures that fail to draw upon 
the information resources on the Internet and elsewhere. Schools that do not provide their 
faculty with classrooms where dynamic audio and visual media are easily used will be un- 
able to attract good faculty and good students. Like business and mdustiy, campuses can- 
not afford to ignore technology. 

This thesis makes no special distinction for “educational” software. In earlier years 
when computer time was scarce and interfaces were primitive, college courses often relied 
on the simplistic interfaces and slow, unreliable algorithms of “student” DSS packages 
whose performance was sufficient for exercises. Today students can be taught with the 
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same packages that practitioners use for applications of realistic size and complexity. 

Many of the packages reviewed are available at low academic prices, with the least expen- 
sive versions being limited to reasonably large number of variables and constraints. In the 
case of eveiy package reviewed, a demonstration copy of the program was available for 
download on the World Wide Web. 

1. Survey Responses 

Among the respondents to the thesis survey, 14 were users who taught DSS at 
both the undergraduate and / or graduate levels. They used a variety of different DSS 
packages in their classes including spreadsheets. The survey asked educators whether 
their use of a DSS package in a decision support class was intended to teach the student 
how to use a DSS or whether it was used to support learning DSS concepts. Respondents 
were mixed in their responses with some leaning heavily towards teaching DSS use and 
others using the programs to support concepts. The overall response was nearly evenly 
split. The selection of a program for the class room will depend in large part on its in- 
tended purpose. Some programs reviewed follow traditional DSS thinking. For example, 
DATA allows the user to create traditional influence diagrams and decision trees. Crite- 
rium Decision Plus uses very traditional AHP diagrams. Either of these programs would 
be good choices for use by a professor whose main purpose is reinforcing DSS concepts. 
Other programs use less traditional methods. Which and Why would be a good choice of 
programs to demonstrate what packages are available but it’s non traditional approach 
makes it less desirable for teaching concepts. Appendbc F provides detailed survey re- 
sponses from educators. 

2. Software Selection 

In the thesis survey educators were asked about five primary criteria to consider 
when selecting DSS software including the length of time required to learn the program, 
the ease of use, the availabihty of multiple platform support including use on a network 
and cost. 
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The length of time required to use a DSS and the closely related criteria, ease of 
use, were ranked as very important by respondents to the survey. These two features ac- 
counted for an average of almost 60 points out of 100 points allotted the selection portion 
of the survey. This is to be expected given the relatively short time which students have to 
learn and use a software package in the classroom and the likely possibility that more than 
one program will be taught. 

We expected multiple platform support would be an important criteria given the 
variety of platforms typically used in schools. Out of 100 points available in the selection 
portion of the survey, however, educators allotted only 3 points to cross platform support 
and ten points to network use. This is attributable to the growing percentage of WINTEL 
based computers and the decreasing numbers of Apple Macintosh. 

Not surprisingly, the cost of acquiring a package represented 3 1 of the 100 points 
available in this section. The DSS packages reviewed here range in price from $20.00 to 
over $10,000. In every case, there are trial versions of the programs evaluated available 
for download from the World Wide Web, however, some are quite limited in their useful- 
ness as instructional tools. In some cases, model size is limited, a small drawback for edu- 
cation purposes. In other cases you can’t save or print results, a much bigger drawback. 

3. Application Features 

The thesis looked at a variety of application features which should be useful to us- 
ers in education. The survey recipients were asked to rate between user interface, data 
handling features and model features. Each of these areas are further subdivided as dis- 
cussed below. 

Important features of the user interface include mouse and Windows support, input 
characteristics and output representation. Respondents allotted twenty points to the ne- 
cessity of having mouse and Windows support. Respondents allocated only 9 points to 
input characteristics but gave 16 points to output. The quality of both these items varies 
greatly from program to program. 
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Educators gave relatively few points to the importance of a packages data handling 
characteristics. Database management functions and import export capabilities each re- 
ceived about 10 points while version support received only 4 points. For the purposes of 
classroom use, it is unlikely that users will require data import or export since they will 
likely be working with small demonstration data sets. Likewise, it is unlikely that there 
will be many versions of a given database. 

Collectively, the ability to build a variety of models which may be large and com- 
plex, gained 25 out of 100 points. Some of the programs which model uncertainty will 
build both influence diagrams and decision trees while others work only with the later. 

The programs which use multi-criteria decision making use AHP and in some instances 
SMART. As with the model management features, version support was of little impor- 
tance to educators. This is to be expected as version support is more important in col- 
laborative efforts such as might be found in business. The ability to export models to 
other applications was also important, earning 17 points. Most of the DSS software re- 
viewed did a poor job of export. 

4. Decision Support Features 

The survey asked educators to allocate points between five DSS related features, 
the ability to model uncertainty, use subjective data, enter utility functions, determine risk 
and perform sensitivity analysis. Each of these features was deemed equally important 
with uncertainty and sensitivity analysis receiving slightly more preference. The programs 
reviewed perform these function in a variety of ways. Some programs will allow the user 
to enter subjective data while others require numeric entries. Most of the programs allow 
some form of sensitivity analysis. Some, such as LDW, allow the user to enter statistical 
data where actual data isn’t available and then display uncertainty in decisions. 

5. Support and Training Features 

Educators were asked to rate the importance of built-in training fiinctions includ- 
ing the availability of sample exercises and tutorials and whether the program provides ex- 
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planations and insight. Most respondents felt that the availability of sample exercises and 
tutorials were very important giving those features 27 and 33 points respectively. All of 
the programs come with sample problems. 21 points were allocated to the importance of 
the program explaining results with 19 points being assigned to the importance of the pro- 
gram providing insight into recommendations. 

The DSS reviewed offer various methods of support for users. Educators were 
asked to rate the importance of six support features including: an online Ust of frequently 
asked questions, availability of discussion groups and mail lists, support via phone or 
email, the level of local support and the availability of online documentation. The avail- 
ability of online documentation was rated as the most important feature. The other five 
features each received equal ratings. 

D. DEMONSTRATION 

Desktop systems are quite useful at the prototyping stage, where the work is fo- 
cused on building an acceptable model and demonstrating sufBciently promising results to 
justify a larger investment. To some extent, educators also build demonstration systems 
which are used to support a lecture on a particular type of decision making. 

Depending on the type of ultimate model planned, any of these programs might be 
useful for prototyping. The first feature required is that the program support the type of 
model ultimately planned for development. The length of time required to construct a 
problem is also important as is a flexible UIMS which can simulate the full scale product. 
Of less importance is a full featured DBMS since small demonstration data sets will 
probably be used. 

E. VARIATIONS ACROSS USE CATEGORIES 

There were some significant differences in what was important to educators and to 
production users as well as many similarities. When rating selection criteria, educators 
were most concerned about the time it would take for their students to learn the package 
as well as the cost of the package. This only makes sense considering the high cost of 
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some software packages and the limited amount of time available for a student to master a 
program. Production users not surprisingly were most interested in the “power” of a 
packages. They too were interested in ease of use and learning time, though less so than 
educators. 



1. Application Features 

Both educational users and production users assigned roughly the same number of 
points to features in this area. Surprisingly, neither placed much emphasis on the data 
handling capabilities of the programs, placing emphasis instead on the user interface fea- 
tures and the ability to build large complex models. 

2. DSS Features 

There were some differences in DSS features educators and production users 
found important. Educators allotted three times as much emphasis to the importance of a 
programs ability to model utility functions. In a pure academic setting, this would be de- 
sirable in a program since the professor could interactively demonstrate this important 
concept. In a production setting, often the manger may prefer to simply express his pref- 
erence for one choice over the next, leaving the details of utility to the underlying pro- 
gram. 

Production users ranked the importance of being able to measure uncertainty and 
sensitivity more strongly than did educators. Although educators are interested in being 
able to demonstrate these features in the classroom, production users must have this in- 
formation to make informed decisions. Thus the higher ranking. 

3. Support and Training 

In the area of support and training, educators for obvious reasons were more inter- 
ested in online tutorials than were production users. Educators were also more interested 
in bdng able to receive support through online discussion groups and mail lists while pro- 
duction users preferred to be able to pick up the phone and call someone. Both produc- 
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tion and educational users weighted the importance of online documentation highly. De- 
signers, however, gave this relatively little priority, apparently preferring to provide writ- 
ten documentation. 

F. SUMMARY 

Different DSS features are more important to some users than others. Designers 
of DSS software not surprisingly, place more emphasis on the production use of their pro- 
grams than on education use. As such, there priorities are more closely aligned with pro- 
duction users than educators. 

In a production setting, DSS are perhaps more practical for non-recurring strategic 
decisions. The limited data import and export capabiUties of today’s products make them 
less suited for recurring decisions which may rely on external data. DSS can make excel- 
lent supplements to courses of instruction which teach DSS. They can be used either to 
teach DSS or to demonstrate interactively the concepts of decision trees or multi-criteria 
decision making. Table 3 summarizes production and educational users priorities when 
using DSS. The following chapter reviews some of the leading DSS packages demon- 
strating the current “state of the art” as it relates to the feature set discussed in this and the 



previous chapter. 



t Judged important 
4 Judged less important 


Production Users 


Educational Users 


All Users 


Selection considera- 
tions 


tPower 

iCost 


tLength of time to leam 
tCost 


tWell designed DIMS 
tEase of use 
4-Multiple platform sup- 
port 


Application features 




t Ability to build differ- 
ent types of models 
tModel Export 


tAbility to build com- 
plex models 


DSS Features 


tSensitivity analysis 
tModeling uncertainty 






Training 


tExplanation of results 
and insight 


tTutorials 


t Sample exercises 


Support 


tPhone support 


tOnline documentation 





Table 3 
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V. DESKTOP DESCISION ANALYSIS SOFTWARE 



A. INTRODUCTION 

Having discussed the feature set which define a desktop decision support system, 
this chapter provides an overview of currently available packages. This thesis will not at- 
tempt to review the packages in depth. New releases are being introduced too fast to 
make such an assessment of lasting value. The popular press contains numerous reviews 
of such packages, some of which are listed in the bibliography. This review will concen- 
trate on how the currently available packages meet or do not to meet the requirements set 
forth in the preceding chapters. Table 4 shows the packages reviewed and the producer. 
Appendix B contains a chart detailing more specific information on these programs. 



Software Title 


Company 


@RISK, TopRank 


Palisade Corp. 


AliahTHINK! 


Aliah Inc. 


Criterium Decision Plus 


InfoHarvest Inc. 


Decide Right for Windows 


Avantos Performance Systems Inc. 


Decision Analysis by TreeAge (DATA) 


TreeAge Software Inc. 


Decision Pro 


Vanguard Software 


DPL 


Applied Decision Analysis 


Expert Choice Professional 


Expert Choice Inc. 


Logical Decisions 


Logical Decisions 


Which & Why 


Arlington Software Corp. 



Table 4 

B. TECHNICAL CONSIDERATIONS 



None of the packages reviewed possessed unusual hardware requirements. A few 
require 8 Mb’s of RAM but most will run with 4 Mb’s. All packages will run in the Win- 
dows 3.1 environment; many also have versions designed for Windows 95. Several still 
have DOS versions available. A few are available for Macintosh. Technical considera- 
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have DOS versions available. A few are available for Macintosh. Technical considera- 
tions are a go, no go situation with which most users are familiar. Appendix B shows a 
matrix summarizing the programs requirements. 

All of the packages listed below are devoid of security features such as password 
protection of files and model designs. Even where intended for use in network situations, 
there is no protection against unintentional or unauthorized modification. 

C. USER INTERFACE MANAGEMENT SYSTEM 

The user interface of the programs reviewed vary greatly though all are relatively 
easy to learn and to use. The figures in this section show some of the user interfaces used 
to set up problems and make comparisons. 

There is considerable difference in the quality of the displays of desktop DSS soft- 
ware. Some adhere closely to the principles of visual design whereas other pay little at- 
tention. 



1. Building Decision Problems 

Figures 9 through 15 show some of the variety of input methods offered by the 
packages reviewed. For the most part, desktop DSS offer the user few options for con- 
structing decision problems. Most make use of Windows pull-down menus and toolbox 
buttons which allow the user to “construct” a problem on a clean screen. A few of the 
programs allow the user to start with a “brainstorming” feature. This feature allows the 
user to list random thoughts in a “note pad” area of the screen, then arrange these 
thoughts hierarchically and finally convert the list to a decision matrix. 
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Figure 9 Criterium Decision Plus uses a traditional AHP graph to de- 
sign the model. Values can be entered using either pairwise compari- 
son or absolute scores. The user can select either AHP or SMART as 
the analysis method. [From Decision Plus by Info Harvest] 




Figure 10 Which & Why uses slide bars to set up a pmrwise comparison. 
Once decision criteria are prioritized, the user selects subjectively between 
the different outcome variables as they relate to each criteria. A difficulty 
with this method is that if the user is weighing a number of alternatives, he 
must remember the score assigned to a given data value. [From Which and 
Why by Arlington Software] 
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Figure 1 1 Expert Choice uses a matrix based on AHP to set up the decision 
model. [From Expert Choice by Expert Choice] 




Figure 12 Once the decision model is set up in Expert Choice, users can 
chose several methods of weighing alternatives, a matrix based method 
shown on the left, or a graphical method, on the right. [From Expert 
Choice by Expert Choice] 
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Figure 13 Decide Right uses a less traditional methodology. In ranking 
criteria, the user drags boxes corresponding to the different criteria to- 
wards the top left as they become more important, [From Decide Right by 
Avantos Performance Systems] 
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Figure 14 DATA uses influence diagrams to formulate the decision 
model. Once the diagram is drawn, the user then uses pop up boxes to 
enter probabilities and values about the various nodes. The influence 
diagram may then be converted into a decision tree. DPL works in a 
similar manner. [From DATA by Tree Age] 
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Figure 15 In Decision Pro the users starts with the decision tree view. Us- 
ers build the tree node by node, entering data values and probabilities as 
they go. [From Decision Pro by Vanguard Software] 

2. Outputs 

Outputs from the programs have more in common than the overall interfaces. The 
basic output of all of the AHP based programs is a bar chart showing a ranking of outputs 
as shown in Figure 16. Most programs also offer a stacked bar chart which shows the 
contribution from each of the criteria. Figure 17 shows this type of chart. Decide Right 
will print a “plain English” decision report and export it to a word processor for editing. 
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Figure 16 Criterium Decision Plus and most programs provide a bar chart 
which shows overall score of each output variable. [From Decision Plus by 
InfoHarvest] 
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Figure 17 Several of the programs provide a breakdown of which criteria 
contributed most to the output. This allows the decision maker to deter- 
mine which criteria contribute most to the various outcomes. [From LDW 
by Logical Decisions] 
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Programs offer a variety of ways to measure sensitivity, an essential ingredient of a 
good DSS. Figure 7 shows the traditional sensitivity chart. Some programs offer the 
useful feature of dynamic sensitivity as shown in FigurelS. 
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Figure 18 Some programs offer dynamic sensitivity analysis. In this pro- 
gram, as the user adjusts the criteria weights (shown on the left) using the 
mouse, the output recommendations (on the right) adjust accordingly. This 
allows useful insight into the importance of different input variables. [From 
Expert Choice by Expert Choice] 

Scatter diagrams. Figure 19, and spider (or Radar) diagrams. Figure 20, provide 
the user additional ways of reviewing data. 
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Figure 19 Some multi-criteria programs produce a scatter diagram which 
allow the decision maker to compare two variables for each alternative. 
[From Expert Choice by Expert Choice] 




Figure 20 One program produces a Spider (or Radar) chart which shows 
weights assigned to different criteria. [From AliahThink by Aliah] 



Decision tree based programs such as DATA or Decision Pro produce risk profile 
graphs as shown in Figure 2 1 . This graph shows the user the range and likelihood of out- 
comes. 
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Figure 21 Programs which use decision making under uncertainty provide 
risk profile graphs. These graphs show the predicted likelihood of the 
various possible outcomes. [From Decision Pro by Vanguard] 

3. Macros and Programming 

With the exception of the spreadsheet add-ins, none of the programs provide for 
macros though AliahThink does offer a form of programming which allows the user to 
write customized routines. All of the programs allow models to be built and saved and 
then run by other user thus allowing a sort of “expert knowledge base." None of these 
programs work well with automated routines such as a user might write with Delphi or 
Visual Basic though an experienced programmer could force the programs to work with 
considerable effort. 

D. DATA MANAGEMENT SYSTEM 

The programs reviewed offer fairly weak DBMS features. In most cases, once 
data is entered the user is unable to perform features such as search and query or to ex- 
tract subsets of the data. None of the programs offer version support. As an alternative, 
however, all programs allow the user to use the “save as” feature to save different data 
sets and rankings. 
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As discussed in Chapter HI, Section D.3.b., a key factor to consider when select- 
ing the system is the type of data which will be used. If the decision problem being solved 
uses absolute vice relative values, the choice of programs is limited to those which will ac- 
cept actual data. LDW and Decision Plus are examples of programs which allow the user 
to enter numerical data and assign utility functions. Other programs, such as Which and 
Why and Decide Right force the user to make pairwise comparisons between the different 
decision variables. This method essentially forces the user to determine a utility function 
value. Not being able to enter data is a limitation in some cases, however, for a simple 
decision where the user doesn’t wish to spend time gathering data, a subjective compari- 
son may suffice. For example, if ranking the desirability of the fighter aircraft with a top 
speed of mach 1.5, the user might choose a score of eight of ten or nine of ten where as a 
plane with a speed of mach 1 .4 might be assigned a score of seven or eight. A significant 
draw back to this method is that when entering large amounts of data, the user might be 
inconsistent in assigning scores. 

Some of the programs offer data import and export capabilities. None perform the 
task in a completely straight forward manner, however, most will create a comma delim- 
ited file and some will save data in other formats such as Microsoft Excel or Lotus 123. 
Table 5 shows a sample export table from Expert Choice. Numbers indicate the corre- 
sponding utility value of each criteria. Table 6 shows a similar output produced by LDW. 
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Car 


PRICE 




0.16447 




GRAND AM 


0.04785 




MAXIMA 


0.02784 




MERCEDES 


0.00899 




VOLVO 


0.01022 




THUNDER 


0.06957 



Table 5 Sample export table showing utility values assigned to price. 
[From Expert Choice by Expert Choice] 



ALTERNATIVES 








NAME 


Price 


Power 


Fuel Economy 


Styling 


Chevy S-10 


10000 


140 


16.5 


Ugly 


Mitsubishi 


11000 


109 


21.25 


Mediocre 


Toyota 


15000 


125 


21 


Mediocre 


Ford Ranger 


14000 


160 


17.5 


Attractive 


E>odge Ram-50 


9000 


109 


23 


Muscular 


Dodge Dakota 


17500 


175 


15.5 


Nondescript 


NOMORE 










ALTERNATIVE UTILnTES 








NAME 


Best Truck 


Cost 


Performance 


Styling 


Chevy S-10 


0.433423794 


0.72134414 


0.055692126 


5 


Mitsubishi 


0.715351046 


0.824600757 


0.600205822 


3.848591549 


Toyota 


0.465027273 


0.3 


0.721473789 




Ford Ranger 


0.420115691 


0.322454794 


0.521369844 


1 


Dodge Ram-50 


0.691738966 


0.733333435 


0.61926907 


2 


Dodge Dakota 


0.250020476 


0.148733611 


0.392523047 


3 


NOMORE 











Table 6 Sample export tables showing data entered into program (top) and 
utility scores (bottom). Unfortunately, these tables contain only numeric 
data, not the formulas which were used to calculate the numbers. [From 
LDW by Logical Decisions] 



E. MODEL MANAGEMENT SYSTEM 

The model management system of the DSS reviewed are fairly rigid which is ex- 
pected of a DSS generator. The programs take different approaches to solving decision 
problems. Among the programs which solve decisions under uncertainty, all use decision 
trees to solve the problem. DATA and DPL offer the user the choice of starting with an 
influence diagram. Decision Pro starts with the decision tree. Of the programs which 
solve multi-criteria decisions, the programs fall into two broad categories, those that use 



fairly traditional methods such as AHP or SMART and those which use unique decision 
models. LDW or Decision Plus are examples of programs which use traditional decision 
analysis models while Decide Right and Which and Why use their own models. None of 
the programs offer you the ability to export models to other applications. Some of the 
programs offer a form of version support. DATA and Decide Right for instance, allow 
the user to save different versions of a model and then recall them later. In most program 
the user must use the “save as” command to save different versions of the model as differ- 
ent file names. 

F. GROUP FEATURES 

Though the programs reviewed for this thesis are not categorized as group deci- 
sion support systems, a few do offer group DSS features. Which and Why allows multiple 
users to rank their preferences and combines these preferences into a group decision. 
Different weights can be assigned to different decision makers. Decide Right allows the 
user to enter different scenarios which can be compared graphically. Group features may 
be desirable, especially in higher level decision making where decisions are frequently 
made by group consensus. 

G. SUPPORT 

Program support and instruction are provided in several manners including help 
files, tutorials, documentation, and various modes of operation. All of the programs come 
with sample problems included in the documentation. 

Online help varies from program to program. Where as some offer only basic in- 
formation about the main features of the program, some offer a virtual course in DSS. 
Criterium Decision Plus, for instance, discusses in detail AHP and SMART, the decision 
making process and how to analyze output. Decision Pro offers a similar level of detail 
for problems involving decision making under uncertainty. Others programs, such as 
LDW or DATA, offer only basic information about the program features. A very useful 
feature found in some programs is the presence of a HELP buttons on individual windows 
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as shown in Figure 22 This allows the user to step back for a moment and refresh then- 
knowledge on a given feature. 




Figure 22 The figure on the left has a help button on each window in case 
the user wants more help on the given feature. The figure on the right 
doesn't. [From Decision Plus by Info Harvest and LDW by Logical Deci- 
sions] 



Some of the programs have online computer tutorials, and all come with sample 
problems which the user can work through. Expert Choice’s tutorial includes onscreen 
movies which show the novice user how to use the program. Some have tutorials which 
can be run on the World Wide Web. 

None of the programs offer different expert and novice modes of operations 
though in Decide Right, the user can “turn off’ pop up screens which offer instructions 
once they are familiar with the program. Notwithstanding, many of the programs offer the 
user different methods of accomplishing the same task such as pull down menus for the in- 
experienced user to control-key functions for the more experienced. 

Most of the vendors provide some degree of consulting services and on sight 
trmning and decision coaching. Vendor Web sites discuss the level of services available. 
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H. 



SPREADSHEET ADD-mS 



Spreadsheet add-ins offer an alternative method of conducting analysis. Though 
not traditional DSS, these programs bring a greater degree of analysis capability to a pro- 
gram, the spreadsheet, with which many are already familiar. 

The techniques of risk analysis provide powerful tools to decision makers in man- 
aging decisions subject to uncertainty. Using the traditional spreadsheet, users can enter 
single point estimates of variables and predict a single output. By manipulating the vari- 
ables, the users can look at a range of outputs, however, with multiple variables this can 
quickly become a burdensome task. A limited number of trials with varied data, or a non 
systematic approach can lead to an output which provides little insight. Spreadsheet add- 
ins attempt to solve this problem. These programs allow users to vary input to perform a 
number of simulations and to show a range of potential outcomes. Input can be varied in 
a variety of methods including multiple point estimates and over 20 statistical distribu- 
tions. The output can then be viewed in a number of ways including distribution graphs, 
sensitivity graphs, and text output. 

Perhaps the biggest advantage of these programs is that they run inside a spread- 
sheet thus making full use of the spreadsheets powerful interface and programming capa- 
bilities. All spreadsheet features are available including data import and export, object 
embedding and linking (OLE), macros, and more. What is perhaps most important is that 
most users are familiar with spreadsheets eliminating the need to learn to use an entirely 
new package. Figures 23 through 26 illustrate the features of the spreadsheet add-in. 
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Figure 23 The user starts in the traditional spreadsheet. In this example a 
company is starting a new product line. The bottom line figure is the net 
present value of the cash flow over ten years. The initial price of the new 
product and the start up costs will be varied using various statistical distri- 
butions. Note the added toolbar buttons. The user selects fields to vary 
(shown in red) and then starts the simulation. [From @Risk by Palisade] 
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Figure 24 The program produces a variety of statistics for the resulting 
output. [From @Risk by Palisade] 



3 7HBPI 



Distribution for NPV 10%/C32 




Values in Thousands 



Figure 25 This graph shows the probability distribution for the net present 
value in this problem. This give the decision maker much greater insight 
than a single point estimate. [From @Risk by Palisades] 
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Figure 26 This graph shows the potential range of values of cash flow over 
the subsequent ten year period [From @Risk by Palisade] 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

The primary focus of the software vendors continues to be on analysis features 
rather than support for non-analysts in structuring the problem. These analysis features 
are useful to all users and critical for the insight that decision makers need. However, the 
result is that there are relatively few people that are qualified and comfortable in using 
these packages. These packages will never become as widespread as spreadsheets and 
databases as long as the user has to be a highly educated analyst. [Booed, 1996] 

Users currently have the choice of two levels of software, those designed for 
someone with experience in DSS and those with less experience. Programs such as LDW 
and Decision Plus, as well as the decision making under uncertainty based programs, as- 
sume that the user has considerable knowledge in decision methods. Other programs, 
such as Which & Why and Decide Right, use a less traditional techniques for reaching de- 
cisions. These programs, however, require little DSS knowledge from the user and essen- 
tially guide the decision maker through making the decision relying on intuitive interfaces. 
The type of software most appropriate in a given situation depends on the users needs. 

There are a variety of excellent programs available which offer differing ap- 
proaches to decision making. As discussed in this thesis, a careful match between the us- 
ers needs and the features offered by these program should provide a product which will 
enhance the decision making or educational process. 

B. RECOMMENDATIONS 

This thesis touched only briefly on group support features which are becoming 
much more common. Further research on these program in the format of this thesis would 
be beneficial. This thesis did not focus on real world scenarios where use of a desktop 
DSS generator would be beneficial. Research involving the practical use of these power- 
ful programs in DoD would be useful. 
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APPENDIX A - METHODS FOR DECISION MAKING 

There are a number of techniques which have been developed to assist the manager 
in making decisions. Though we typically solve these problems with the aid of a computer 
program, they do not necessarily require the use of a computer. These models serve not 
only to present an answer to a problem but to allow the decision maker to develop an in- 
sight into the problem itself by examining various facets of it and by exploring alternative 
ways of looking at the problem. 

When using decision analysis, a problem is broken down into clearly defined com- 
ponents which depict all options, outcomes, inputs and probabilities. These values are 
quantified thus permitting the evaluation of costs and benefits associated with alternative 
courses of action. This process does not replace the decision maker with arithmetic, 
rather, it provides an orderly and more easily understood structure that helps to support 
the decision maker by providing him -with logically sound techniques to support and ensure 
internal consistency of his judgments. 

There are several popular models in use in decision analysis including the decision 
tree, influence diagram and the analytical hierarchy process. 

A. INFLUENCE DIAGRAMS 

Among the most basic decision analysis tools is the influence diagram. Influence 
diagrams consist of blocks which represent decisions, chances and outcomes and the rela- 
tionship which exists between these events. Figure 27 shows a decision problem con- 
cerning how to invest. The influence diagram does not solve the problem but allows the 
decision maker to visualize the decision and the factors which influence the outcome. 
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Figure 27 DATA uses influence diagrams to allow the decision 
maker to formulate the decision problem. In the diagram, blue 
squares represent decisions, green circles represent chance oc- 
currences and the red diamond represents the outcome. [From 
Decision Analysis by Tree Age] 



B. DECISION TREES 

Decision trees offer a way to structure and represent a problem and choose the 
course of action consistent with certain kinds of objectives. In order to use a decision 
tree, the decision maker must know, or be able to estimate the possible actions or choices, 
the cost or value of each action, and how likely each action is. Given this information, the 
decision tree returns an expected value of each path from which the decision maker will 
chose the best path. 

Decision trees consist of two types of nodes, chance and choice. Chance nodes 
represent events outside the decision makers control and involve a percentage likelihood 
that the given event will occur. Choice nodes represent decisions which must be made. 
Figure 28 shows a decision tree which depicts the investment decision shown as an influ- 
ence diagram in Figure 27. In this scenario, outcomes (profit) are listed on the right. De- 
cision nodes are shown as blue boxes. There are two choices for the decision maker, to 
invest in CD’s pa)dng 5% or in a risky investment. Chance nodes are shown as green cir- 
cles with the percentage likelihood of each eventuality shown, as well as the expected 
value of the overall node. The preferred path based on expected value is shown in ma- 
genta. 
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Figure 28 This decision tree was generated by DATA and shows a decision 
to drill for oil. [From Decision Analysis by Tree Age] 

Decision trees represent problems at an elemental detail level. Problem can 
quickly grow to hundreds of nodes. 

C. MULTI-CRITERIA DECISION MAKING 

Multiple-criteria decision making is applicable to decision problems with a decision 
maker considering multiple alternatives whose objective function values are known with 
certainty. Consider the following example. You are choosing between two fighter air- 
craft, the details of which are shown in Table 7. 





Aircraft A 


Aircraft B 


Price 


$17 million 


$19 million 


Sneed 


Mach 1.7 


Mach 1.8 


■Ranee 


800 miles 


700 miles 



Table 7 



In this example, the decision maker must decide if aircraft A, which has a lower 
price but lower performance figures, is better than aircraft B, a higher performance but 
more expensive aircraft. The decision support system must transform the scores of the 
different criteria to comparable values or scales which may or may not be linear. 

There are several issues to consider. In this example, the criteria are on different 
scales. Is a speed of mach 1 .7 and a range of 800 miles better or worse than a speed of 
mach 1.8 and a range of 700 miles? In order to evaluate this situation, we need to trans- 
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form these scores to comparable values or scales. Another problem arises from the fact 
that even for individual criteria, the utility fimctions may not be linear. Is the 100 mile 
difference between a range of 700 and 800 miles as important as the difference between 
300 and 400 miles? In order to resolve these issues, it is necessary assign new scales of 
measurement using some upper and lower bound and then evaluate the weighted scores 
for each alternative. 

D. ANALYTICAL HIERARCHY PROCESS (AHP) 

The analytical hierarchy process is a theory of measurement for dealing with deci- 
sion problems characterized by multiple objectives, criteria and multiple alternatives. The 
AHP method focuses on preparing a hierarchical design and conducting an evaluation 
which emphasizes the expertise of the decision maker given a set of alternatives. In the 
design phase, the decision maker breaks the problem into constituent parts and develops a 
hierarchy of objectives, criteria and alternatives. In the aircraft example the objective is to 
procure the best aircraft for the dollar. The criteria include speed, range and a host of 
other features not shovm. Our alternatives include aircraft A and B. In the evaluation 
phase, the decision maker rates the alternatives on each criteria and develops relative 
weights using pairwise comparison. Each element in a level is compared in relative terms 
for each criteria just above the element. The output from this is a relative scale of meas- 
urements of the priorities or weights of elements. Figure 29 29 shows one program's ap- 
proach to developing a solution to our aircraft problem using AHP. 

There are several considerations involved in using AHP including what kind of hi- 
erarchy should be used and what should go into each level. The designer must consider 
the scale and measurements to use. Inconsistencies can develop due to the use of relative 
scales or absolute measurements (e g., good, average, poor). 



70 




Figure 29 Criterium Decision Plus utilizes AHP in decision making. [From 

Decision Plus by Criterium] 

E. SIMPLE MULTIATTRIBUTE UTILITY TECHNIQUE (SMART) 

Another ranking technique is the Simple Multiattribute Utility Technique 
(SMART) which is available in Decision Plus and LDW. 

Before the user can synthesize the contributions of criteria with differing scale, the 
decision model must provide a method that allows the user to handle model scales on an 
equal footing. For example, in the aircraft selection problem above, the decision maker 
needs to assign comparable scales to speed and range. In both AHP and SMART, the 
model technique itself handles this problem. This is achieved by normalization, where all 
scales are converted to a common internal scale that takes a value between 0 and 1 . 

In AHP, to get the effective importance of a subcriterion, the program takes its 
user-given score divided by the sum of the scores of all the other subcriteria of the com- 
mon criterion. This guarantees that no matter what the original score, all weights will fall 
between 0 and 1 . 

This is a very simple and effective method that allows integration of units that are 
completely different. However, it is clearly a relative judgment method. How a particular 
alternative scores in the model depends on what other alternatives are being considered. 
For instance, in AHP if you created a model for one set of alternatives, adding a new al- 
ternative later will change the scores of the original alternatives, sometimes in surprising 
ways. 
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SMART does not use a relative method for scaling units to a standard scale (from 
0 to 1). Instead, the user can define their own method for doing this using a value func- 
tion. A value fimction allows decision makers to explicitly define how each value on the 
scale is transformed to the common model scale. 

F. UTILITY THEORY 

Utility theory offers an approach for making and understanding decisions. It is 
based on a subjective view of how valuable certain payoffs are when outcomes are either 
certain or probabilistic and is a measure of the worth one attaches to an outcome or situa- 
tion. Utility is subjective and may change over time. Rational decision makers will at- 
tempt to maximize utility. Utility functions are frequently expressed graphically. Figure 
30 shows the utility value for given speeds in our aircraft problem. 




Figure 30 Criterium uses a graphical approach to assigning utility value. 

The user uses the mouse to adjust the curve to the appropriate utility value. 

[From Decision Plus by Criterium] 

In this example, utility value remains relatively low until the plane’s speed reaches 
mach one. A fighter aircraft with a speed below mach one has little utility regardless if the 
speed is mach . 1 or mach .5. The utility increases rapidly up until about mach 1.5. Above 
this, little additional utility need be assigned because the plane is already faster than most 
of our competitors. By converting a raw data value, mach 0 to mach 2, to a utility score 
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of zero to one, we can now use this number can now be used in comparison with other 
factors involved in our decision. 



G. WEIGHTING AND UTILITY FUNCTIONS 



All DSS allow the decision maker to assign weights to the criteria being measured. 
Most use a graphical approach utilizing some form of pairwise comparison which allows 
the user to rank their preference of criteria over another. Figure 3 1 shows one example 



where the user uses slide bars to rank each feature on a ten point scale. 




Figure 3 1 Which and WTiy uses a graphical method to assign values to 
preferences. In this example, the user uses slide bars to rank his pref- 
erence for one feature over another. The pie chart shows the overall 
preference weights. [From WTiich and Why by Arlington Software] 



H. MONTE CARLO SIMULATION 

Monte Carlo simulation is used for modeling uncertainty to help manage risk and 
simulate complex systems. Risk is generally thought of as either the chance of some un- 
fortunate event happening or as the volatility of a key performance measure such as prof- 
its. Decision trees provide an excellent method of planning uncertain actions to take in re- 
sponse to events. Monte Carlo simulations, on the other hand, are best for modeling un- 
certainty and volatility. 

Monte Carlo simulation allows the user to replace uncertain quantities in the model 
with the results of a number of random trials and then see how that uncertainty affects the 
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results. Like decision trees, Monte Carlo simulations result in an expected value that aids 
in choosing the most attractive course of action. They also provide information about the 
range of outcomes such as best- and worst -case, probability of reaching specific targets, 
most likely outcomes, etc. 

One advantage of Monte Carlo simulation is that it is easy to apply. When com- 
bining several uncertain values, determining the uncertainty on the result can be very com- 
plex. For example, when adding two uncertain values, the uncertainty on the result is 
somewhat less that the sum of the original uncertainties. Using Monte Carlo simulation, 
this and similar effects are handled automatically so the user doesn't need to know much 
about statistics to get accurate results. 
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APPENDIX B - TECHNICAL FEATURES SUMMARY 
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SPREADSHEET ADD-INS 
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User can create custom interface. 
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Shows sensitivity on the input screen vice as an output graph. 
Will create a text and graphics report for use in word processor. 
Can utilize spreadsheet for required math functions. 
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Excellent demonstration CD ROM available. 



APPENDIX C - EDUCATOR SURVEY 

Dear Sir, 

I am a graduate student in the Information Technology Management program at 
the Naval Postgraduate School in Monterey, California. Along with Professor Hemant 
Bhargava and Professor Sridhar, I am preparing a Master's thesis titled "An Analysis of 
Software for Decision Analysis". 

As you may know, there are now various commercial "DSS Generators" for working 
with decision analysis methods such as decision trees and multicriteria decision makin g 
Broadly, we have classifted such software into 3 categories of use: production (development 
of real DSS af^Ucations), demonstration (development of prototype plications) and educa- 
tion (in DSS methods and/or software). 

The focus of my thesis is to examine the feature set required to suprt DSS work in 
the above 3 categories, to compare this against feature sets available in leading commercial 
systems, and to develop general recommendations for selecting software for the 3 categories 
of use. 



In order to determine which characteristics of a DSS package are most significant in 
educational use, w e have developed the following list of features we feel would be important 
to educators. However, I would greatly aR)reciate it if you could take a few minutes to re- 
view the list and answer the questions posed. In addition, I would appreciate any other input 
you may have regarding what is important to you when selecting DSS software for use in 
your classes. 

1. Basic questions 

a. At what level do you instruct DSS [Undergrad, Graduate, Continuing education 
programs. Multiple]: 

b. Which DSS packages have you used before in the classroom? With regards to 
the questions shown below, what features attracted you to the packages you used? 

c. Which DSS packages have you rejected for use in the classroom? Why? 

In EACH section below you have a total of 100 points which may be distributed 
among the various choices. Allocate more points to the features you consider most impor- 
tant. The sum of points allocated IN EACH SECTION should equal 100. 

2. In your courses, what is your emphasis with regards to using DSS software 
(allocate 100 points between each of the following two items): 

a. How to use DSS software 

b. Using DSS software as support for teaching DSS 

3. Software selection. How important are the following issues when selecting soft- 
ware to use (allocate 100 points between each of the following five items): 

a. Length of time to required for student to learn 

b. Ease of use 

c. Multiple platform support (MAC, DOS) 
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d. Ability to use on network 

e. Cost of acquiring package 

f. Please give any other remarks related to what is important to you in selecting 

software: 

4. Application related questions. Which available features are important (allocate 
100 points between each of the following ten items): 

User Interface: 

a. Mouse and windows suiqwrt 

b. Input characteristics 

c. Output representation (reports, graphs) 

Data handling features 

d DBMSftmctions 

e. Version management 

f. Data import and export capability 

g. Models (A variety of models on a single package) 

h. Ability to build large, complex models 

i. Version support 

j. Model export to other applications 

k. Availability of statistical ftmctions 

l. Please give any other remarks related to what is important to you in regarding 
application design: 

5. Training related issues. Which of the following features related to using the 
program are important (allocate 100 points between each of the following four items): 

a. Sample exercises are provided 

b. Computer based tutorials are provided 

c. Results are explained 

d The in'ogram provides tips and insight 

e. Please give any other remarks related to what is important: 

6. How important is it that the program sui^rt the following DSS considerations 
(allocate 100 points between each of the following five items): 

a. Modeling uncertainty 

b. Subjectivity 

c. Utility fimctions 

d. Risk 
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e. Sensitivity analysis 

f. What other items are important to you regarding features of DSS; 

7. Siqqwrt. How important are the following support related issues, (allocate 100 
points between each of the following six items); 

a. FAQ file online 

b. Discussion group availability 

c. Mail lists availability 

d. Technical support via phone or email 

e. Level of local support available 

f Is there online documentation 

g. Are there other support features that are important to you: 

8. Please take a moment and think about your use of DSS software in the class- 
room. Are there any other considerations you would like to share: 
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APPENDIX D - VENDOR SURVEY 



Dear Sir, 

I am a graduate student in the Information Technology Management 
program at the Naval Postgraduate School in Monterey, California. Along 
with Professor Hemant Bhargava and Professor Sridhar, I am preparing a 
Master's thesis titled "An Analysis of Software for Decision Analysis". 

As you know, there are now various commercial "DSS Generators" for 
working with decision analysis methods such as decision trees and multicriteria 
decision making. Broadly, we have classified such software into 3 categories 
of use: production (development of real DSS applications), demonstration 
(development of prototype applications) and education (in DSS methods 
and/or software). 

The focus of my thesis is to examine the feature set required to support 
DSS work in the above 3 categories, to compare this against feature sets avail- 
able in leading commercial systems, and to develop general recommendations 
for selecting software for the 3 categories of use. 

In order to determine which characteristics of a DSS package are most 
significant to users of such programs, we have developed the following list of 
features we feel are important to all users of DSS software. However, I would 
greatly appreciate it if you could take a few minutes to review the list and an- 
swer the questions posed from the perspective of the user as you see his needs. 
Any will be used for research purposes only. You confidentiality will be en- 
sured 

1. Basic questions 

How many copies of your program have you sold? 

How long has your product been on the market? 

c. As well as producing software, do you offer consulting services? 

In EACH section below you have a total of 100 points which may be 
distributed among the various choices. Allocate more points to the features 
you consider most important. The sum of points allocated IN EACH 
SECTION should equal 100. 

2. In designing your software, who do you see as your primary in- 
tended user (allocate 100 points between each of the following four items): 

a. Education 

Production (Strategic use) 
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Production (lower level use) 

d. Demonstration 

3. Software selection. When designing your software, how important 
do you feel the following issues are to your users when they select software 
(allocate 100 points between each of the foUowing five items); 

Length of time to required for user to learn 

b. Ease of use 

c. Multiple platform support (MAC, DOS) 

d. Ability to use on network 

e. Cost of acquiring package 

f Power (variety of features, outputs, size of models) 

g. Please give any other remarks related to why your users select a 
particular package: 

4. Application related questions. When designing your product, which 
available features do you feel are most important to your users (allocate 100 
points between each of the following ten items): 

User Interface; 

a. Mouse and windows support 

b. Input characteristics 

c. Output representation (reports, graphs) 

Data handling features 

d. DBMS fimctions 

e. Version management 

f Data import and export capability 

Models 

g. Ability to build large, complex models 

h. Model version support 

i. Model export to other applications 

j. Availability of statistical functions 

k. Please give any other remarks related to what is important to you in 
regarding application design: 
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5. Training related issues. When designing your product, which of the 
following features related to using the program are import.ant to your users 
(allocate 100 points between each of the following four items): 

a. Sample exercises are provided 

b. Computer based tutorials are provided 

c. Results are explained 

e. The program provides tips and insight 

e. Please give any other remarks related to what is important with re- 
gards to training: 

6. When designing your product, how important is it to your users that 
the program support the following DSS considerations (allocate 100 points 
between each of the following five items): 

a. Modeling uncertainty 

b. Subjectivity 

c. Utility functions 

d. Risk 

e. Sensitivity analysis 

Availability of math functions to document decisions 

g. What other items are important to you regarding features of DSS: 

7. Support. How important are the following support related issues to 
your users, (allocate 100 points between each of the following sbc items): 

a. FAQ file online 

b. Discussion group availability 

c. Mail lists availability 

d. Technical support via phone or email 

e. Level of local support available 

f Is there online documentation 

g. Are there other support features that are important to you: 

Please take a moment and think about other considerations when de- 
signing DSS software. Are there any other issues you would like to share: 



. 95 



9. If you have the names and email addresses of any corporate users 
who you believe would be willing to answer questions such as the ones in this 
survey I would appreciate your sharing them with me? 
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APPENDIX E - PRODUCTION USERS SURVEY 

Dear Sir, 

I am a graduate student in the Information Technology Management program 
at the Naval Postgraduate School in Monterey, California. Along with Profes- 
sor Hemant Bhargava and Professor Sridhar, I am preparing a Master's thesis 
titled "An Analysis of Software for Decision Analysis". 

As you know, there are now various commercial "DSS Generators" for work- 
ing with decision analysis methods such as decision trees and mukicriteria deci- 
sion making. Broadly, we have classified such software into 3 categories of 
use: production (development of real DSS applications), demonstration 
(development of prototype applications) and education (in DSS methods 
and/or software). 

The focus of my thesis is to examine the feature set required to support DSS 
work in the above 3 categories, to compare this against feature sets available in 
leading commercial systems, and to develop general recommendations for se- 
lecting software for the 3 categories of use. 

In order to determine which characteristics of a DSS package are most signifi- 
cant to users of such programs, we have developed the following list of fea- 
tures we feel are important to all users of DSS software. However, I would 
greatly appreciate it if you could take a few minutes to review the list and an- 
swer the questions posed from the perspective of the user. Any will be used 
for research purposes only. You confidentiality will be ensured 

1 . Basic questions 

a. Which DSS packages have you used before in the classroom? With 
regards to the questions shown below, what features attracted you to the pack- 
ages you used? 

b. Which DSS packages have you rejected for use in the classroom? 

Why? 

In EACH section below you have a total of 100 points which may be distrib- 
uted among the various choices. Allocate more points to the features you con- 
sider most important. The sum of points allocated IN EACH SECTION 
should equal 100. 

2. What do you see as the primary intended purpose of DSS software (allocate 
100 points between each of the following four items): 

a. Production (Strategic use) 

b. Production (lower level use) 
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3. Software selection. When selecting software, how important do you feel 
the following issues are (allocate 100 points between each of the following five 
items): 

a. Length of time to required for user to learn 

b. Ease of use 

c. Multiple platform support (MAC, DOS) 

d. Ability to use on network 

e. Cost of acquiring package 

f Power (variety of features, outputs, size of models) 

g. Please give any other remarks related to why your users select a 
particular package: 

4, Application related questions. Which available features do you feel are 
most important (allocate 100 points between each of the following ten items): 

User Interface: 

a. Mouse and windows support 

b. Input characteristics 

c. Output representation (reports, graphs) 

Data handling features 

d. DBMS fimctions 

e. Version management 

f Data import and export capability 

Models 

g. Ability to build large, complex models 

h. Model version support 

i. Model export to other applications 

j. Availability of statistical fimctions 

k. Please give any other remarks related to what is important to you in 
regarding application design: 

5. Training related issues. Which of the following features related to using 
the program are important (allocate 100 points between each of the following 
four items): 
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a. Sample exercises are provided 

b. Computer based tutorials are provided 

c. Results are explained 

e. The program provides tips and insight 

e. Please give any other remarks related to what is important with re- 
gards to training: 

6. How important is it that the program support the following considerations 
(allocate 100 points between each of the following five items): 

a. Modeling uncertainty 

b. Subjectivity 

c. Utility fimctions 

d. Risk 

e. Sensitivity analysis 

f Availabihty of math fimctions to document decisions 

g. What other items are important to you regarding features of DSS: 

7. Support. How important are the following support related issues (allocate 
100 points between each of the following six items): 

a. FAQ file onhne 

b. Discussion group availability 

c. Mail lists availability 

d. Technical support via phone or email 

e. Level of local support available 

f Is there online documentation 

g. Are there other support features that are important to you: 

8. Please take a moment and think about other considerations when selecting 
or using DSS software. Are there any other issues you would like to share: 

9. If you have the names and email addresses of any corporate users who you 
believe would be willing to answer questions such as the ones in this survey I 
would appreciate your sharing them with me? 



99 



I 

I 

I 



100 



(O 

a: 



< 

o 

3 

Q 

LU 



(O 

UJ 

(O 



o 

0. 

(O 

UJ 

q: 

>• 

UJ 

> 

0^ 

3 

(O 

■ 

u. 

X 

Q 

z 

UJ 

0. 

0. 

< 



_a> 

3 

<v 

E 

cd 

Um 

CtO 
O 

Um 

cx 
c 

.9 o 



S 

o 



0^ 

3 

O' 



« 

n 

n 

-o 



cd 



^ < 



.2 

3 



cd 



c/3 

<U 



cd 



cd 

<u 

c 

o 

c/3 

c/3 

c 

O 

c/3 

3 

cr 

<u 



c/3 

•o 

cd 

GO 

<U 



cd 
o 
3 

<U 
GO 
.£ 

3 .ti 

S ^ 

c^- 

£ 
o 
o 



c 

o 

U 



B ^ Cd 

rrs 



Cd 

3 

-3 

3 

o 

'3*' 

cd 

Um 

GO 

3 

T3 

C 

D 

c/^ 

c/^ 

Q 



c/3 

C 

3 

O 

o 

TD 

U 

> 

0> 



- Q 



c/3 

C/3 

jS 

o 



o 



x> 

•3 

3 

c/3 

3 

3 

|1 

" I 

<u o 

GO 

3 c« 

^ <D 

O GO 

3 Cd 

cu ^ 
00 ^ 
00 cx 



c^- 



c^- 

E 

o 

o 

c/3 

C/3 

3 



O 

<u 

j3 



•J2 

^ § 

. 

^ -3 
<U 
-♦-* 
o 
cd 



cd 



O 

’t-* 

c/3 

'♦— » 
c 

*o 

cx 

3 

O 

E 

<u 

cd 

o 

^ o 
o 

c/3 

<1^ -3 
o Cd 



O 

o 

c/3 

3 

O 



3 

3" 

3 

2 

3 

O 



cd 

^ 2 

c U 

2 W 
E ^ 

X 

Ss 

3 ^ 

J w 

Is 

ho -o 
^ cd 

E cd 



c/3 ^ 



.3 o 



<u 

c/3 

3 



-3 

0> 

O 

3 

‘ 5 ? 

3 

O 

0> 



c/3 

o 

3 

a< 

CO 

CO 

Q 

o 



o 



cd 



c 

^ a 
o t: 



Cd 

3 

-C 

3 

O 

3 

-3 

C 

o 



o 



ffi 

u 

< 

PQ 



3 

O-H 

3 
C ^ 



GO 

C 

2 

0> 

-C 

C4-^ 

O 

-C 

o 

cd 

3 

c 

<1> 

<1> 



0> 

c/3 

C 

*o 

cx 

o 

o 



c 

-g *o 
^ cx 

c/3 <4-4 

■5 E 

s 

8 2 



o 

o. 

E 

*s 

o 

£ 

fe 

2 

*co 

C 

o 

o 

3 

o 



o 

4—* 

cd 

o 

o 



3 

cd 

O 

c/3 

CO 

CO 

Q 

GO 

C 

3 



c/3 

•3 

3 

GO 

3 



c/3 

3 



3 

3 

O 



3 

-C 

c/T 

3 

c/3 

U. 

3 

O 

o 

3 

O 

C c« 

- £ 

<u 



o 

csi ^ 



d> 

cd 

cC 

o 

c/3 

CO 

CO 

Q 

a> 

C/3 

3 



o 

a: 

cd 



CO 

CO 



Q 



GO 

C 

2 

o 

cd 

o> 



t: 

o 

cx 

o< 

s 

c/3 

cd 

0> 

3 

<tJ 

o 

c/3 

CO 

CO 

Q 

GO 

.£ 

D 






101 



c4 


90 


O 


80 


65 


40 


O 


30 


90 


20 


50 


80 


o 


95 


60 


Average 50.4 49.6 

Standard Deviation 31.3 31.3 


« 


o 


09 


20 


35 


60 


06 


70 


o 


80 


50 


O 

<S 


001 




40 


u 


NexpertObject - too hard to leam and get access to, 
QSB - old, user unfriendly system, CMMS from 
Erikson&Hall book - too restrictive, DOS based. 


Packages that became obsolete 


Excel (LP, MIP), IThink (dynamic modeling), several hypermedia system (where I developed my 
own DSS generator 


I've rejected other packages primarily for cost rea- 
sons 


Oracle express. Too expensive 


Sigma - not common software 


Most specialized packages: expense, difficulty to 
leam. 




More industrial strength expert systems. To hard to 
leam in time available 


The most important features are cost and core 
functionality. 






SLAM II. Too much time to teach technicalities 
which have no instructive value 


Delphi 


US 

?H 


Excel, 'home-made' MCDM written in Turbo 
Prolog VPExpert, NexpertObject, Knowledge- 
Pro - for postgrads 


Lotus, SAS 


Expert Choice, Exsys, Group Systems, Man- 
agement Decision Assistant, spreadsheets 


Excel. Easy to use known interface. Oracle 
express. Exclusive packages 


ubiquity and friendliness 


spreadsheet, mainly (MS Excel) comprehen- 
sive, cheap, can illustrate most DSS concepts. 


Demos (currently called Analytica) 


IFPS, VPExpert Light learning curve. Build 
models quickly, sensitivity analysis, uncer- 
tainty 


DPL, Criterium, Expert Choice 




ESS 


SSS. GAMS 


Doesn't believe you can package DSS. Must 
build in context of its organizational use. 


CQ 


Both 


Grad 


Grad 


Undergrad 


Grad 


Undergrad 


Both 


Grad 


Grad 


Grad 


Both 


iBoth 


Both 


Both 



102 



Question 3 

Software selection. How important are the following issues when selecting software to use 
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Question 4 

Application related questions. Which available features are important? 
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Question 5 

Training related issues. Which of the following features related to using the program are important 
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Question 6 

How important is it that the program support the following DSS considerations 
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Question 7 

Support. How important are the following support related issues. 
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APPENDIX G - SURVEY RESPONSES - VENDORS 
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Question 3 

Software selection. When designing your software, how important do you feel the following issues are to your users when they select 
software 
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Question 4 

Application related questions. When designing your product, which available features do you feel are most important to your users 
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Question 5 

Training related issues. When designing your product, which of the following features related to using the program are important to 
your users 
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Question 6 

When designing your product, how important is it to your users that the program support the following DSS considerations 
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Question 7 

Support. How important are the following support related issues to your users. 
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Question 8 

Please take a moment and think about other considerations when designing DSS software. Are there any other issues you would like 
to share 
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Question 2; 

What do you see as the primary intended purpose of DSS software 
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Question 3 

Software selection When selecting software, how important do you feel the following issues are 
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Question 4 

Application related questions. Which available features do you feel are most important 
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Question 5 

Training related issues. Which of the following features related to using the program are important 
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Question 6: 

How important is it that the program support the following considerations 



V 

- 

- 'H 

M 

M 

s6 














The variance between one application and another 
is too high to reasonably address this question. It 
depends on the problem at hand 




26 9 8 16 28 16 Average 

12 9 9 10 11 15 Standard Deviation 




Availability of math 
functions 


o 


O 




00 


o 

cn 




o 


y-- 


Sensitivity analy- 
sis 








00 


in 

(N 




o 




Risk 


o 


O 

<N 




00 


in 

CM 




m 


u 


Utility func- 
tions 


o 


in 




00 


O 




m 




Subjectivity 


in 


in 

CM 




00 


o 




o 


VD 


Modeling un- 
certainty 


O 

(N 


in 

CM 




00 


o 

CM 




o 



122 



Question 7: 

Support. How important are the following support related issues 
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APPENDIX I - COMPARISON OF USER CONSIDERATIONS 
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